Part 1 of a tutorial series on writing C extensions for Python 3. This part begins with teaching how to write Hello World to stdout using printf() from a C method implementation.

Introduction

Python provides many great tools for writing fast extensions in C or C++. Here, we will be learning how to call a method that prints "Hello World". But, we will be writing the code that does the printing in C! But first, we need to get our environment set up. I will be running this tutorial on an Ubuntu 15.10 64-bit virtual machine, but this tutorial will be easily translatable to pretty much any linux distribution. You may need to adapt package manager commands and package names to your system, however.

Environment

So, we need an environment to compile our extension in. Let's use a virtual environment. Start by making sure you have the virtualenv command available on your machine. If not, run sudo apt-get install virtualenv to get it. Next, we're going to create a virtual environment for Python 3.4 to do our testing in.

Additional prerequisites are

python3.4
python3.4-dev
build-essential

Make sure you have these packages installed to avoid confusing error messages during compilation.

mkdir python-c-extension && cd python-c-extension
virtualenv -p /usr/bin/python3.4 env
source env/bin/activate

Now, you should see (env) prefixed to your terminal prompt. This means we're working inside a python virtual environment so we won't be screwing around with any system level packages. In other words, we don't have to worry about messing up any other part of our machine, as long as we stick to Python 3.4.

Build Setup

OK, so we're virtualized, and we have an environment suitable for building python extensions. Let's get to writing some code! First of all, we need to create a setup.py script to tell Python what our module is, what it's source files are, and some other metadata. That file is going to look like this:

setup.py:

from distutils.core import setup, Extension

hello_world_module = Extension('hello_module',
                           sources = ['hello_module.c'])

setup(name = 'lesson_one',
      version = '1.0',
      description = 'Python Package with Hello World C Extension',
      ext_modules = [hello_world_module],

      url='http://adamlamers.com',
      author='Adam Lamers',
      author_email='adamlamers at gmail dot com')

To break down what this file means, lets look at it's core pieces:

hello_world_module contains the definition of an ext_module that we pass to setup. In it's definition, we tell it that the extension's name is going to be hello_module, and that it's list of source files (we don't have one yet but we're going to call it hello_module.c) only contains one file in the current directory.

Then, we call setup, which is a distutils function that defines a python package. We tell it the name of our python package (lesson_one), our version number, a description, and a list of extension modules that we want to include in this package. Also, I threw in some other arguments to suppress warnings when running setup.py sdist.

The Good Stuff

Now, we need to actually write the code that the setup file will compile into a module. Let's start by looking at the final product:

hello_module.c:

#include <Python.h>
#include <stdio.h>

//Actual module method definition - this is the code that will be called by
//hello_module.print_hello_world
static PyObject* hello_module_print_hello_world(PyObject *self, PyObject *args)
{
    printf("Hello World\n");
    Py_RETURN_NONE;
}

//Method definition object for this extension, these argumens mean:
//ml_name: The name of the method
//ml_meth: Function pointer to the method implementation
//ml_flags: Flags indicating special features of this method, such as
//          accepting arguments, accepting keyword arguments, being a
//          class method, or being a static method of a class.
//ml_doc:  Contents of this method's docstring
static PyMethodDef hello_module_methods[] = { 
    {   
        "print_hello_world",
        hello_module_print_hello_world,
        METH_NOARGS,
        "Print 'hello world' from a method defined in a C extension."
    },  
    {NULL, NULL, 0, NULL}
};

//Module definition
//The arguments of this structure tell Python what to call your extension,
//what it's methods are and where to look for it's method definitions
static struct PyModuleDef hello_module_definition = { 
    PyModuleDef_HEAD_INIT,
    "hello_module",
    "A Python module that prints 'hello world' from C code.",
    -1, 
    hello_module_methods
};

//Module initialization
//Python calls this function when importing your extension. It is important
//that this function is named PyInit_[[your_module_name]] exactly, and matches
//the name keyword argument in setup.py's setup() call.
PyMODINIT_FUNC PyInit_hello_module(void)
{
    Py_Initialize();

    return PyModule_Create(&hello_module_definition);
}

See? That's not too bad! (At least for C.) Now, lets look at what each of these definitions means in detail. Let's go bottom up, since the farther down the code you go, the more dependencies there are above it.

First up, the PyMODINIT_FUNC PyInit_hello_module(void) definition. By looking at it, we can kind of tell that it's the initialization function for this particular module. We see it returns PyMODINIT_FUNC, which is an abbreviation of Python Module Initialization Function. Inside of it's definition, we call a function named Py_Initialize(), which performs the interpreter's internal state configuration in a way that our module can see it. This must be the first call your module makes (with a few minor exceptions.) Then, we return the value of PyModule_Create()back to some unknown place. We have to assume here that the Python interpreter is calling this function, so essentially what we are returning is an object representing the module in a way that can be understood by Python.

Next, we see that PyInit_hello_module refers to a structure named hello_module_definition, which is of the type PyModuleDef. This is a special data structure that defines the name, description, and methods of the python module. Pretty self explanatory there. The -1 is the argument to m_size, which is out of the scope of this tutorial. It basically allows you to not use global state for your module's state, but we don't care about that here.

Inside hello_module_definition, there is a reference to hello_module_methods. This structure contains a list of every method we want to be visible through to Python in our module. Here, we only have one method. print_hello_world. In the method definition for print_hello_world, the arguments tell Python what the name of the method is, the location of the method's implementation in C, what type of method it is using flags, and the docstring that describes it. We see here that it refers to the C function hello_module_print_hello_world, which prints "Hello World" to stdout, and then uses a special macro to return a None value to the Python caller. This is shorthand for incrementing the reference count of Py_None and then returning it. i.e.

Py_INCREF(Py_None);
return Py_None;

So, that explains what our C code is all about, now let's install the module, and see if it does what we want it to do. Let's install it in our virtual environment and give it a run.

Testing

We're going to use pip to install the module into a virtual environment. Make sure you've got your venv activated, then run

pip install .

inside the directory with setup.py and hello_module.c. If all goes well, pip should report that it

Successfully installed lesson-one

Then open up your Python interpreter and call the method that is defined in C code:

Python 3.4.3+ (default, Oct 14 2015, 16:03:50) 
[GCC 5.2.1 20151010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello_module
>>> hello_module.print_hello_world()
Hello World

It works! Now you can combine the speed of C with the convenience of Python.