Protocol Buffers and Thrift are good formats for data serialization, but they are less popular in dynamic languages like Python than schemaless alternatives like JSON because of the need to run a code generator on the schema file to create the necessary classes. CodeGenLoader is a Python import hook that automatically and transparently runs the necessary code generator at import time. It includes support for both Thrift and Protocol Buffers, with an extensible base class that can be used to add more code generators.

All you need is two lines of magic in an __init__.py:

import codegenloader.protobuf
__path__ = codegenloader.protobuf.make_path(__name__, ".")

Get it with pip install codegenloader or check out the source on github.

2to3 can be a useful tool for developing Python libraries that work with both Python 2 and 3, but it’s fallen out of fashion because it intrudes into the development process in awkward ways and generally slows things down. In spite of its weaknesses, I believe 2to3 is a good approach for many libraries to support multiple versions of Python, so in this post I’ll share the approach and tools I used to add Python 3 support to Tornado

2to3 vs single-source

The major alternative to 2to3 is to use a single source tree written to the common subset of all supported versions of Python. This is a viable approach even for large projects, as shown by e.g. Vinay Sajip’s work with Django. However, the resulting code requires many compatibility shims, and looks somewhat unnatural from the perspective of both Python 2 and 3. This is especially true if compatibility with Python 2.5 is desired – many features that ease the transition to Python 3 were introduced in version 2.6. Personally, I prefer the workflow afforded by 2to3, in which the source code remains more-or-less normal Python 2, but it works on Python 3 as well.