Faster Python with Cython and PyPy: Part 1

Tom Craven, Technical Lead

For several years now, Python has been my go-to language for web and IoT projects, owing to its readable syntax, flexibility, and comprehensive library support. From a programmer’s perspective, it is a truly enjoyable language to read and write, and I’ve experienced significant productivity gains when building systems with Python over languages like Java and C#.

Is Python Really That Slow?

No language is perfect of course, and one criticism often leveled at Python (more specifically CPython, the Python interpreter that executes Python code) is that it is slow compared to C, C++, and Java. It is indeed true that some algorithms written in Python can execute 10-100 times(!) slower than the same algorithm implemented in C; typically the more numerical calculations and loops that are involved, the slower the Python version becomes. In real life applishutterstock_288042365cations, especially those that are I/O bound—like web servers that spend most of their time waiting for the database—the lack of speed is often not noticeable, and the cost is far outweighed by productivity benefits of developing your system in Python.

It is also the case that performance-critical parts of Python libraries are typically implemented in C and are therefore no slower than their C counterparts. Every now and again however, a library is not optimized enough, or a program runs too slowly for our liking. In these cases there are several compelling options available to speed up Python programs. I recently spent some time experimenting with two of the most popular tools: Cython and PyPy. In this two-part post, I will discuss both solutions, starting today with Cython.

Speeding Things Up with Cython

Cython is an optimizing static compiler for the Cython language, which is a superset of Python. The Cython language supports calling C functions and declaring C types on variables, function arguments, and return values. Cython programs are actually Python programs that run on the standard CPython interpreter. They differ from pure Python programs in that they import compiled Cython modules written in the Cython language.

CPython supports C extension modules, which allow performance-critical modules to be written in C, call C library functions, and make system calls. Extension modules are rather complicated to write however, and require programmers to manage Python’s memory explicitly inside their C code. Cython makes the programmer’s job easier by creating these C extension modules automatically from Cython source code and abstracting away memory management and low-level error handling.

A Look at Cython Code

cython codeThe Cython build process first translates the Cython source code into optimized C code, which is a CPython extension module. It then uses a standard C compiler to compile the module into a C shared object (.so) file, which can be directly imported by Python programs via the import statement. Cython programs can run at C-like speeds owing to their optimized and compiled nature. I’ve introduced some Cython code here in this image, and in-depth introductions to Cython can be found on Wikipedia and the Cython website. Next week, we will talk about PyPy and contrast it with Cython to find our winner in the race for a faster Python.

 

Cardinal Peak
Learn more about our Audio & Video capabilities.

Dive deeper into our IoT portfolio

Take a look at the clients we have helped.

We’re always looking for top talent, check out our current openings. 

Contact Us

Please fill out the contact form below and our engineering services team will be in touch soon.

We rely on Cardinal Peak for their ability to bolster our patent licensing efforts with in-depth technical guidance. They have deep expertise and they’re easy to work with.
Diego deGarrido Sr. Manager, LSI
Cardinal Peak has a strong technology portfolio that has complemented our own expertise well. They are communicative, drive toward results quickly, and understand the appropriate level of documentation it takes to effectively convey their work. In…
Jason Damori Director of Engineering, Biamp Systems
We asked Cardinal Peak to take ownership for an important subsystem, and they completed a very high quality deliverable on time.
Matt Cowan Chief Scientific Officer, RealD
Cardinal Peak’s personnel worked side-by-side with our own engineers and engineers from other companies on several of our key projects. The Cardinal Peak staff has consistently provided a level of professionalism and technical expertise that we…
Sherisse Hawkins VP Software Development, Time Warner Cable
Cardinal Peak was a natural choice for us. They were able to develop a high-quality product, based in part on open source, and in part on intellectual property they had already developed, all for a very effective price.
Bruce Webber VP Engineering, VBrick
We completely trust Cardinal Peak to advise us on technology strategy, as well as to implement it. They are a dependable partner that ultimately makes us more competitive in the marketplace.
Brian Brown President and CEO, Decatur Electronics
The Cardinal Peak team started quickly and delivered high-quality results, and they worked really well with our own engineering team.
Charles Corbalis VP Engineering, RGB Networks
We found Cardinal Peak’s team to be very knowledgeable about embedded video delivery systems. Their ability to deliver working solutions on time—combined with excellent project management skills—helped bring success not only to the product…
Ralph Schmitt VP, Product Marketing and Engineering, Kustom Signals
Cardinal Peak has provided deep technical insights, and they’ve allowed us to complete some really hard projects quickly. We are big fans of their team.
Scott Garlington VP Engineering, xG Technology
We’ve used Cardinal Peak on several projects. They have a very capable engineering team. They’re a great resource.
Greg Read Senior Program Manager, Symmetricom
Cardinal Peak has proven to be a trusted and flexible partner who has helped Harmonic to deliver reliably on our commitments to our own customers. The team at Cardinal Peak was responsive to our needs and delivered high quality results.
Alex Derecho VP Professional Services, Harmonic
Yonder Music was an excellent collaboration with Cardinal Peak. Combining our experience with the music industry and target music market, with Cardinal Peak’s technical expertise, the product has made the mobile experience of Yonder as powerful as…
Adam Kidron founder and CEO, Yonder Music
The Cardinal Peak team played an invaluable role in helping us get our first Internet of Things product to market quickly. They were up to speed in no time and provided all of the technical expertise we lacked. They interfaced seamlessly with our i…
Kevin Leadford Vice President of Innovation, Acuity Brands Lighting
We asked Cardinal Peak to help us address a number of open items related to programming our systems in production. Their engineers have a wealth of experience in IoT and embedded fields, and they helped us quickly and diligently. I’d definitely…
Ryan Margoles Founder and CTO, notion