Skip main navigation

Creating Cython modules

In this article we show how to turn Python modules into Cythonized ones.
© CC-BY-NC-SA 4.0 by CSC - IT Center for Science Ltd.

Normally, when working with Cython one does not Cythonize the whole program but only selected modules.

Let’s have a look at a very simple Python module that we could store in my_module.py:

def add(x, y):
result = x + y
return result

The module would then be used from some other Python code for example as:

from my_module import add

z = add(4, 5)

Cython can transform the Python code into an equivalent C-code utilizing the
Python API as:

$ cython3 my_module.py

The result is a file my_module.c, which can then be compiled into a
Python C-extension. One can investigate the C-file, but it is not really
meant for humans to read, and the length of the file is over 3000 lines!

Normally, Cython language extensions are used in the modules so that they are
no longer valid Python. Therefore one normally uses .pyx suffix in the
file names of Cython modules to indicate this, i.e. my_module.py would be
made into my_module.pyx.

In order to make building of C-extensions from Cython easier, one does not
normally use Cython directly from the command line but via Python distutils
setup.py build file:

from distutils.core import setup
from Cython.Build import cythonize

setup(
ext_modules=cythonize("my_module.pyx"),
)

In larger real-world applications, this setup file could contain several
modules.

One can then create the C-extension module with:

$ python3 setup.py build_ext --inplace
running build_ext
building 'my_module' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c my_module.c -o build/temp.linux-x86_64-3.6/my_module.o
gcc -pthread -shared ...

where the --inplace option places the C-extension in the current directory.
The end result is a .so file containing the C-extension that can be imported
and used just the same as the pure Python module:

from my_module import add

z = add(4, 5)

As the C-extension implements the fully dynamic Python code (just using the Python C-API), transforming the pure Python module into C-extension gives normally only very modest speed-ups. However, as we will discuss in the following steps, by adding Cython language extensions into the code (so it is no longer valid Python code) it is possible to achieve much more significant performance improvements.

© CC-BY-NC-SA 4.0 by CSC - IT Center for Science Ltd.
This article is from the free online

Python in High Performance Computing

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education