Matt McC Matt McC - 1 year ago 99
C++ Question

What files are required for Py_Initialize to run?

I am working on a simple piece of code that runs a Python function from a C/C++ application. In order to do this I set the PYTHONPATH and run initialize as follows:


Then I import my module and run my function. It works great.

I am now attempting to build an installer for my colleagues to run my code. I want to minimize the number of files I need to include in this installer, for obvious reasons.

Googling around the subject tells me that I should be able to include the files "Python27.lib" and "Python27.dll", then zip the "DLLs" and "Lib" folders and include them. However, when I attempt this, Py_Initialize fails.

A quick examination of what is causing this failure shows that Py_Initialize appears to depend upon a number of .pyc files in the Lib folder including (but not limited to warnings.pyc, _abcoll.pyc, _future_.pyc and the contents of the "encodings" folder.

I cannot understand why this would be. Any advice?

Answer Source

At the beginning I wanted to say that there's no module required (at least no non-builtin one) for Py_InitializeEx, so the only requirement was python27.dll (btw: python27.lib is not required, unless your colleagues want to link something against it - but that wouldn't be very easy w/o Python's Include dir). I had this code (BTW: I am using Python 2.7.10 that I built using Visual Studio 10.0):

#include <stdio.h>
#include <conio.h>
#include <Python.h>

int main() {
    int i = 0;
    char *pyCode = 
        "s = \"abc\"\n"
        "print s, 1234";
    i = PyRun_SimpleString(pyCode);
    printf("PyRun_SimpleString returned: %d\nPress a key to exit...\n", i);
    return 0;

It built fine, it ran OK from Visual Studio, and from the command line (after copying the .dll in its folder). But then I copied the .exe and .dll to another computer and when running, bang!!!

ImportError: No module named site

Considering that:

  • I have no PYTHON* env vars set in neither of the consoles on the 2 machines where I ran the .exe (with different results)
  • On both machines the Python installation is on the same path (I modified it on the machine that doesn't work)

I don't know why it doesn't behave the same (one thing that I haven't check is that there might be some registry key on the machine that works?).

Note: site is a (.py(c)) module located under %PYTHON_INSTALL_DIR%\Lib.

Then, I browsed Python's source code and I came across this (file: pythonrun.c, line: 269, function Py_InitializeEx or pythonrun.c:269:Py_InitializeEx - this is how I'm going to refer a point in the source code):

    if (!Py_NoSiteFlag)
        initsite(); /* Module site */

while in pythonrun.c:727:initsite:

    m = PyImport_ImportModule("site");

which is pretty obvious (Py_NoSiteFlag is 0).

Then I noticed that Py_NoSiteFlag is declared as extern __declspec(dllexport), so I modified my code to:

#include <stdio.h>
#include <conio.h>
#include <Python.h>

extern __declspec(dllimport) int Py_NoSiteFlag;

int main() {
    int i = 0;
    char *pyCode = 
        "s = \"abc\"\n"
        "print s, 1234";
    Py_NoSiteFlag = 1;
    i = PyRun_SimpleString(pyCode);
    printf("PyRun_SimpleString returned: %d\nPress a key to exit...\n", i);
    return 0;

and it works! Yay!

So, at this point only the .dll is required in order to run a piece of code. But I imagine that your code is "a little bit" more complex than that (it has imports). To solve the import problem, you can use this nice module called modulefinder (part of Python 2.7's standard modules). To make use of it:

  • save the code that you execute from C in a .py file
  • run modulefinder against it

Here's an example for my code (pyCode contents in my C program, saved in a file called

s = "abc"
print s, 1234

Running %PYTHON_INSTALL_DIR%\python %PYTHON_INSTALL_DIR%\Lib\ yields:

Name File
---- ----
m __main__

But, if I add an import os (which is a pretty common module) statement in the file, the above command yields:

Name File
---- ----
m UserDict %PYTHON_INSTALL_DIR%\lib\
m __builtin__
m __future__ %PYTHON_INSTALL_DIR%\lib\
m __main__
m _abcoll %PYTHON_INSTALL_DIR%\lib\
m _codecs
m _collections
m _functools
m _hashlib %PYTHON_INSTALL_DIR%\DLLs\_hashlib.pyd
m _heapq
m _io
m _locale
m _random
m _sre
m _struct
m _subprocess
m _threading_local %PYTHON_INSTALL_DIR%\lib\
m _warnings
m _weakref
m _weakrefset %PYTHON_INSTALL_DIR%\lib\
m array
m atexit %PYTHON_INSTALL_DIR%\lib\
m binascii
m cPickle
m cStringIO
m codecs %PYTHON_INSTALL_DIR%\lib\
m collections %PYTHON_INSTALL_DIR%\lib\
m copy %PYTHON_INSTALL_DIR%\lib\
m copy_reg %PYTHON_INSTALL_DIR%\lib\
m difflib %PYTHON_INSTALL_DIR%\lib\
m doctest %PYTHON_INSTALL_DIR%\lib\
m dummy_thread %PYTHON_INSTALL_DIR%\lib\
P encodings %PYTHON_INSTALL_DIR%\lib\encodings\
m encodings.aliases %PYTHON_INSTALL_DIR%\lib\encodings\
m errno
m exceptions
m fnmatch %PYTHON_INSTALL_DIR%\lib\
m functools %PYTHON_INSTALL_DIR%\lib\
m gc
m genericpath %PYTHON_INSTALL_DIR%\lib\
m getopt %PYTHON_INSTALL_DIR%\lib\
m gettext %PYTHON_INSTALL_DIR%\lib\
m hashlib %PYTHON_INSTALL_DIR%\lib\
m heapq %PYTHON_INSTALL_DIR%\lib\
m imp
m inspect %PYTHON_INSTALL_DIR%\lib\
m itertools
m keyword %PYTHON_INSTALL_DIR%\lib\
m linecache %PYTHON_INSTALL_DIR%\lib\
m locale %PYTHON_INSTALL_DIR%\lib\
P logging %PYTHON_INSTALL_DIR%\lib\logging\
m marshal
m math
m msvcrt
m nt
m ntpath %PYTHON_INSTALL_DIR%\lib\
m opcode %PYTHON_INSTALL_DIR%\lib\
m operator
m optparse %PYTHON_INSTALL_DIR%\lib\
m os2emxpath %PYTHON_INSTALL_DIR%\lib\
m pickle %PYTHON_INSTALL_DIR%\lib\
m posixpath %PYTHON_INSTALL_DIR%\lib\
m pprint %PYTHON_INSTALL_DIR%\lib\
m random %PYTHON_INSTALL_DIR%\lib\
m repr %PYTHON_INSTALL_DIR%\lib\
m select %PYTHON_INSTALL_DIR%\DLLs\select.pyd
m shlex %PYTHON_INSTALL_DIR%\lib\
m signal
m sre_compile %PYTHON_INSTALL_DIR%\lib\
m sre_constants %PYTHON_INSTALL_DIR%\lib\
m sre_parse %PYTHON_INSTALL_DIR%\lib\
m stat %PYTHON_INSTALL_DIR%\lib\
m string %PYTHON_INSTALL_DIR%\lib\
m strop
m struct %PYTHON_INSTALL_DIR%\lib\
m subprocess %PYTHON_INSTALL_DIR%\lib\
m sys
m tempfile %PYTHON_INSTALL_DIR%\lib\
m textwrap %PYTHON_INSTALL_DIR%\lib\
m thread
m threading %PYTHON_INSTALL_DIR%\lib\
m time
m token %PYTHON_INSTALL_DIR%\lib\
m tokenize %PYTHON_INSTALL_DIR%\lib\
m traceback %PYTHON_INSTALL_DIR%\lib\
m types %PYTHON_INSTALL_DIR%\lib\
P unittest %PYTHON_INSTALL_DIR%\lib\unittest\
m %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.loader %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.main %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.result %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.runner %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.signals %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.suite %PYTHON_INSTALL_DIR%\lib\unittest\
m unittest.util %PYTHON_INSTALL_DIR%\lib\unittest\
m warnings %PYTHON_INSTALL_DIR%\lib\
m weakref %PYTHON_INSTALL_DIR%\lib\

Missing modules:
? _emx_link imported from os
? ce imported from os
? fcntl imported from subprocess, tempfile
? org.python.core imported from copy, pickle
? os.path imported from os, shlex
? os2 imported from os
? posix imported from os
? pwd imported from posixpath
? readline imported from cmd, pdb
? riscos imported from os
? riscosenviron imported from os
? riscospath imported from os

As you can see, there is an awfully lot of modules (I modified the output a little bit, instead of the actual path I placed the %PYTHON_INSTALL_DIR% env var). In order for the Python code to work, you'll have to include all of those modules/packages in the installer.

Notes about modulefinder's output (that I've noticed while playing with it):

  • It searches for modules recursively, so here is the whole module dependency tree
  • It searches for import statements located in functions (so, not only the ones at module level)
  • It doesn't search for dynamic imports (e.g. __import__)

So, looking at the modules that are required by os, I'm not sure that taking out the site import from C makes much of a difference.

IMPORTANT NOTE: To make sure your .exe works on any computer, you might consider including Visual Studio C runtime library (msvcr##(#).dll) (where # s are placeholders for digits - representing Visual Studio version) in your installer.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download