Skip main navigation

New offer! Get 30% off one whole year of Unlimited learning. Subscribe for just £249.99 £174.99. New subscribers only. T&Cs apply

Find out more

Creating Cython modules

In this article we show how to turn Python modules into Cythonized ones.
Normally, when working with Cython one does not Cythonize the whole program but only selected modules.

Let’s have a look at a very simple Python module that we could store in my_module.py:

def add(x, y):
result = x + y
return result

The module would then be used from some other Python code for example as:

from my_module import add

z = add(4, 5)

Cython can transform the Python code into an equivalent C-code utilizing the
Python API as:

$ cython3 my_module.py

The result is a file my_module.c, which can then be compiled into a
Python C-extension. One can investigate the C-file, but it is not really
meant for humans to read, and the length of the file is over 3000 lines!

Normally, Cython language extensions are used in the modules so that they are
no longer valid Python. Therefore one normally uses .pyx suffix in the
file names of Cython modules to indicate this, i.e. my_module.py would be
made into my_module.pyx.

In order to make building of C-extensions from Cython easier, one does not
normally use Cython directly from the command line but via Python distutils
setup.py build file:

from distutils.core import setup
from Cython.Build import cythonize

setup(
ext_modules=cythonize("my_module.pyx"),
)

In larger real-world applications, this setup file could contain several
modules.

One can then create the C-extension module with:

$ python3 setup.py build_ext --inplace
running build_ext
building 'my_module' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c my_module.c -o build/temp.linux-x86_64-3.6/my_module.o
gcc -pthread -shared ...

where the --inplace option places the C-extension in the current directory.
The end result is a .so file containing the C-extension that can be imported
and used just the same as the pure Python module:

from my_module import add

z = add(4, 5)

As the C-extension implements the fully dynamic Python code (just using the Python C-API), transforming the pure Python module into C-extension gives normally only very modest speed-ups. However, as we will discuss in the following steps, by adding Cython language extensions into the code (so it is no longer valid Python code) it is possible to achieve much more significant performance improvements.

© CC-BY-NC-SA 4.0 by CSC - IT Center for Science Ltd.
This article is from the free online

Python in High Performance Computing

Created by
FutureLearn - Learning For Life

Reach your personal and professional goals

Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.

Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.

Start Learning now