For several years now, Python has been my go-to language for web and IoT engineering projects due to its readable syntax, flexibility and comprehensive library support. From a programmer’s perspective, it is a truly enjoyable language to read and write, and I’ve experienced significant productivity gains when building systems with Python over languages like Java and C#. Despite those advantages, one criticism often leveled at Python (more specifically CPython, the Python interpreter that executes Python code) is that it is slow compared to C, C++ and Java. In this article, we’ll discuss how to speed up Python code with Cython & PyPy.
Contents
Why is Python So Slow?
No language is perfect of course, and it is indeed true that some algorithms written in Python can execute 10-100 times(!) slower than the same algorithm implemented in C. Typically, the more numerical calculations and loops that are involved, the slower the Python version becomes. In real-life applications, especially those that are I/O bound — like web servers that spend most of their time waiting for the database — the lack of speed is often not noticeable, and the cost is far outweighed by the productivity benefits of developing your system in Python.
It is also the case that performance-critical parts of Python libraries are typically implemented in C and are therefore no slower than their C counterparts. Every now and again, however, a library is not optimized enough or a program runs too slowly for our liking. In these cases, there are several compelling options available to speed up Python programs. I recently spent some time experimenting with two of the most popular tools: Cython and PyPy. In this two-part post, I will discuss both solutions, starting today with Cython.
How Cython Speeds Up Python Code
What is Cython? Cython is an optimizing static compiler for the Cython language, which is a superset of Python. The Cython language supports calling C functions and declaring C types on variables, function arguments and return values. Cython programs are actually Python programs that run on the standard CPython interpreter. They differ from pure Python programs in that they import compiled Cython modules written in the Cython language.
CPython supports C extension modules, which allow performance-critical modules to be written in C, call C library functions and make system calls. Extension modules are, however, rather complicated to write and require programmers to manage Python’s memory explicitly inside their C code. Cython makes the programmer’s job easier by creating these C extension modules automatically from Cython source code and abstracting away memory management and low-level error handling.
Take a Look: A Cython Code Example
The Cython build process first translates the Cython source code into optimized C code, which is a CPython extension module. It then uses a standard C compiler to compile the module into a C shared object (.so) file, which can be directly imported by Python programs via the import statement. Cython programs can run at C-like speeds owing to their optimized and compiled nature. I’ve introduced some Cython code here in this image, and in-depth introductions to Cython can be found on Wikipedia and the Cython website.
Next week, we will talk about PyPy and contrast it with Cython to find our winner in the race for a faster Python. Stay tuned!