Nuitka

 

Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, and 3.11. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.

nuitka.net

Nuitka User Manual

Overview

This document is the recommended first read if you are interested in using Nuitka, understand its use cases, check what you can expect, license, requirements, credits, etc.

Nuitka is the Python compiler. It is written in Python. It is a seamless replacement or extension to the Python interpreter and compiles every construct that CPython 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11 have, when itself run with that Python version.

It then executes uncompiled code and compiled code together in an extremely compatible manner.

You can use all Python library modules and all extension modules freely.

Nuitka translates the Python modules into a C level program that then uses libpython and static C files of its own to execute in the same way as CPython does.

All optimization is aimed at avoiding overhead, where it's unnecessary. None is aimed at removing compatibility, although slight improvements will occasionally be done, where not every bug of standard Python is emulated, e.g. more complete error messages are given, but there is a full compatibility mode to disable even that.

Requirements

C Compiler

You need a C compiler with support for C11 or alternatively a C++ compiler for C++03 [1].

Currently, this means, you need to use one of these compilers:

• The MinGW64 C11 compiler, on Windows, must be based on gcc 11.2 or higher. It will be automatically downloaded if no usable C compiler is found, which is the recommended way of installing it, as Nuitka will also upgrade it for you.
• Visual Studio 2022 or higher on Windows [2], older versions will work, but only supported for commercial users. Configure to use the English language pack for best results (Nuitka filters away garbage outputs, but only for English language). It will be used by default if installed.
• On all other platforms, the gcc compiler of at least version 5.1, and below that the g++ compiler of at least version 4.4 as an alternative.
• The clang compiler on macOS X and most FreeBSD architectures.
• On Windows, the clang-cl compiler on Windows can be used if provided by the Visual Studio installer.

[1]

Support for this C11 is given with gcc 5.x or higher or any clang version. The MSVC compiler doesn't do it yet. But as a workaround, as the C++03 language standard is significantly overlapping with C11, it is then used instead where the C compiler is too old. Nuitka used to require a C++ compiler in the past, but it changed.

[2]

Download for free from https://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx (the community editions work just fine). The latest version is recommended, but not required. On the other hand, there is no need to except to support pre-Windows 10 versions, and they might work for you, but support of these configurations is only available to commercial users.

Python

Python Version 2.6, 2.7 or 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11 are supported. If at any moment, there is a stable Python release that is not in this list, rest assured it is being worked on and will be added.

Important

For Python 3.4 and only that version, we need other Python version as a compile time dependency.

Nuitka itself is fully compatible with all listed versions, but Scons as an internally used tool is not.

For these versions, you need a Python2 or Python 3.5 or higher installed as well, but only during the compile time. That is for use with Scons (which orchestrates the C compilation), which does not support the same Python versions as Nuitka.

In addition, on Windows, Python2 cannot be used because clcache does not work with it, there a Python 3.5 or higher needs to be installed.

Nuitka finds these needed Python versions (e.g. on Windows via registry) and you shouldn't notice it as long as they are installed.

Increasingly, other functionality is available when another Python has a certain package installed. For example, onefile compression will work for a Python 2.x when another Python is found that has the zstandard package installed.

Moving binaries to other machines

The created binaries can be made executable independent of the Python installation, with --standalone and --onefile options.

Binary filename suffix

The created binaries have an .exe suffix on Windows. On other platforms they have no suffix for standalone mode, or .bin suffix, that you are free to remove or change, or specify with the -o option.

The suffix for acceleration mode is added just to be sure that the original script name and the binary name do not ever collide, so we can safely overwrite the binary without destroying the original source file.

It has to be CPython, Anaconda Python, or Homebrew

You need the standard Python implementation, called "CPython", to execute Nuitka because it is closely tied to implementation details of it.

It cannot be from the Windows app store

It is known that Windows app store Python definitely does not work, it's checked against.

It cannot be pyenv on macOS

It is known that macOS "pyenv" does not work. Use Homebrew instead for self compiled Python installations. But note that standalone mode will be worse on these platforms and not be as backward compatible with older macOS versions.

Operating System

Supported Operating Systems: Linux, FreeBSD, NetBSD, macOS X, and Windows (32 bits/64 bits/ARM).

Others will work as well. The portability is expected to be generally good, but the e.g. Nuitka's internal Scons usage may have to be adapted or need flags passed. Make sure to match Python and C compiler architecture, or else you will get cryptic error messages.

Architecture

Supported Architectures are x86, x86_64 (amd64), and arm, likely many, many more.

Other architectures are expected to also work, out of the box, as Nuitka is generally not using any hardware specifics. These are just the ones tested and known to be good. Feedback is welcome. Generally, the architectures that Debian supports can be considered good and tested, too.

Usage

Command Line

The recommended way of executing Nuitka is <the_right_python> -m nuitka to be absolutely certain which Python interpreter you are using, so it is easier to match with what Nuitka has.

The next best way of executing Nuitka bare that is from a source checkout or archive, with no environment variable changes, most noteworthy, you do not have to mess with PYTHONPATH at all for Nuitka. You just execute the nuitka and nuitka-run scripts directly without any changes to the environment. You may want to add the bin directory to your PATH for your convenience, but that step is optional.

Moreover, if you want to execute with the right interpreter, in that case, be sure to execute <the_right_python> bin/nuitka and be good.

Pick the right Interpreter

If you encounter a SyntaxError you absolutely most certainly have picked the wrong interpreter for the program you are compiling.

Nuitka has a --help option to output what it can do:

nuitka --help

The nuitka-run command is the same as nuitka, but with a different default. It tries to compile and directly execute a Python script:

nuitka-run --help

This option that is different is --run, and passing on arguments after the first non-option to the created binary, so it is somewhat more similar to what plain python will do.

Installation

For most systems, there will be packages on the download page of Nuitka. But you can also install it from source code as described above, but also like any other Python program it can be installed via the normal python setup.py install routine.

License

Nuitka is licensed under the Apache License, Version 2.0; you may not use it except in compliance with the License.

You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Tutorial Setup and build on Windows

This is basic steps if you have nothing installed, of course if you have any of the parts, just skip it.

Setup

Install Python

• Download and install Python from https://www.python.org/downloads/windows
• Select one of Windows x86-64 web-based installer (64 bits Python, recommended) or x86 executable (32 bits Python) installer.
• Verify it's working using command python --version.

Install Nuitka

• python -m pip install nuitka
• Verify using command python -m nuitka --version

Write some code and test

Create a folder for the Python code

• mkdir HelloWorld
• make a python file named hello.py

def talk(message):
    return "Talk " + message


def main():
    print(talk("Hello World"))


if __name__ == "__main__":
    main()

Test your program

Do as you normally would. Running Nuitka on code that works incorrectly is not easier to debug.

python hello.py

---

Build it using

python -m nuitka hello.py

Note

This will prompt you to download a C caching tool (to speed up repeated compilation of generated C code) and a MinGW64 based C compiler, unless you have a suitable MSVC installed. Say yes to both those questions.

Run it

Execute the hello.exe created near hello.py.

Distribute

To distribute, build with --standalone option, which will not output a single executable, but a whole folder. Copy the resulting hello.dist folder to the other machine and run it.

You may also try --onefile which does create a single file, but make sure that the mere standalone is working, before turning to it, as it will make the debugging only harder, e.g. in case of missing data files.

Use Cases

Use Case 1 - Program compilation with all modules embedded

If you want to compile a whole program recursively, and not only the single file that is the main program, do it like this:

python -m nuitka --follow-imports program.py

Note

There are more fine-grained controls than --follow-imports available. Consider the output of nuitka --help. Including fewer modules into the compilation, but instead using normal Python for it, will make it faster to compile.

In case you have a source directory with dynamically loaded files, i.e. one which cannot be found by recursing after normal import statements via the PYTHONPATH (which would be the recommended way), you can always require that a given directory shall also be included in the executable:

python -m nuitka --follow-imports --include-plugin-directory=plugin_dir program.py

Note

If you don't do any dynamic imports, simply setting your PYTHONPATH at compilation time is what you should do.

Use --include-plugin-directory only if you make __import__() calls that Nuitka cannot predict, and that come from a directory, for everything from your Python installation, use --include-module or --include-package.

Note

The resulting filename will be program.exe on Windows, program.bin on other platforms, but --output-filename allows changing that.

Note

The resulting binary still depends on CPython and used C extension modules being installed.

If you want to be able to copy it to another machine, use --standalone and copy the created program.dist directory and execute the program.exe (Windows) or program (other platforms) put inside.

Use Case 2 - Extension Module compilation

If you want to compile a single extension module, all you have to do is this:

python -m nuitka --module some_module.py

The resulting file some_module.so can then be used instead of some_module.py.

Important

The filename of the produced extension module must not be changed as Python insists on a module name derived function as an entry point, in this case PyInit_some_module and renaming the file will not change that. Match the filename of the source code to what the binary name should be.

Note

If both the extension module and the source code of it are in the same directory, the extension module is loaded. Changes to the source code only have effect once you recompile.

Note

The option --follow-import-to works as well, but the included modules will only become importable after you imported the some_module name. If these kinds of imports are invisible to Nuitka, e.g. dynamically created, you can use --include-module or --include-package in that case, but for static imports it should not be needed.

Note

An extension module can never include other extension modules. You will have to create a wheel for this to be doable.

Note

The resulting extension module can only be loaded into a CPython of the same version and doesn't include other extension modules.

Use Case 3 - Package compilation

If you need to compile a whole package and embed all modules, that is also feasible, use Nuitka like this:

python -m nuitka --module some_package --include-package=some_package

Note

The inclusion of the package contents needs to be provided manually; otherwise, the package is mostly empty. You can be more specific if you like, and only include part of it, or exclude part of it, e.g. with --nofollow-import-to='*.tests' you would not include the unused test part of your code.

Note

Data files located inside the package will not be embedded by this process, you need to copy them yourself with this approach. Alternatively, you can use the file embedding of Nuitka commercial.

Use Case 4 - Program Distribution

For distribution to other systems, there is the standalone mode, which produces a folder for which you can specify --standalone.

python -m nuitka --standalone program.py

Following all imports is default in this mode. You can selectively exclude modules by specifically saying --nofollow-import-to, but then an ImportError will be raised when import of it is attempted at program run time. This may cause different behavior, but it may also improve your compile time if done wisely.

For data files to be included, use the option --include-data-files=<source>=<target> where the source is a file system path, but the target has to be specified relative. For the standalone mode, you can also copy them manually, but this can do extra checks, and for the onefile mode, there is no manual copying possible.

To copy some or all file in a directory, use the option --include-data-files=/etc/*.txt=etc/ where you get to specify shell patterns for the files, and a subdirectory where to put them, indicated by the trailing slash.

To copy a whole folder with all files, you can use --include-data-dir=/path/to/images=images which will copy all files including a potential subdirectory structure. You cannot filter here, i.e. if you want only a partial copy, remove the files beforehand.

For package data, there is a better way, using --include-package-data, which detects data files of packages automatically and copies them over. It even accepts patterns in a shell style. It spares you the need to find the package directory yourself and should be preferred whenever available.

With data files, you are largely on your own. Nuitka keeps track of ones that are needed by popular packages, but it might be incomplete. Raise issues if you encounter something in these.

When that is working, you can use the onefile mode if you so desire.

python -m nuitka --onefile program.py

This will create a single binary, that extracts itself on the target, before running the program. But notice, that accessing files relative to your program is impacted, make sure to read the section Onefile: Finding files as well.

# Create a binary that unpacks into a temporary folder
python -m nuitka --onefile program.py

Note

There are more platform-specific options, e.g. related to icons, splash screen, and version information, consider the --help output for the details of these and check the section Tweaks.

For the unpacking, by default a unique user temporary path one is used, and then deleted, however this default --onefile-tempdir-spec="%TEMP%/onefile_%PID%_%TIME%" can be overridden with a path specification that is using then using a cached path, avoiding repeated unpacking, e.g. with --onefile-tempdir-spec="%CACHE_DIR%/%COMPANY%/%PRODUCT%/%VERSION%" which uses version information, and user-specific cache directory.

Note

Using cached paths will be relevant, e.g. when Windows Firewall comes into play because otherwise, the binary will be a different one to it each time it is run.

Currently, these expanded tokens are available:

Token	What this Expands to	Example
%TEMP%	User temporary file directory	C:\Users\...\AppData\Locals\Temp
%PID%	Process ID	2772
%TIME%	Time in seconds since the epoch.	1299852985
%PROGRAM%	Full program run-time filename of executable.	C:\SomeWhere\YourOnefile.exe
%PROGRAM_BASE%	No-suffix of run-time filename of executable.	C:\SomeWhere\YourOnefile
%CACHE_DIR%	Cache directory for the user.	C:\Users\SomeBody\AppData\Local
%COMPANY%	Value given as --company-name	YourCompanyName
%PRODUCT%	Value given as --product-name	YourProductName
%VERSION%	Combination of --file-version & --product-version	3.0.0.0-1.0.0.0
%HOME%	Home directory for the user.	/home/somebody
%NONE%	When provided for file outputs, None is used	see notice below
%NULL%	When provided for file outputs, os.devnull is used	see notice below

Important

It is your responsibility to make the path provided unique, on Windows a running program will be locked, and while using a fixed folder name is possible, it can cause locking issues in that case, where the program gets restarted.

Usually, you need to use %TIME% or at least %PID% to make a path unique, and this is mainly intended for use cases, where e.g. you want things to reside in a place you choose or abide your naming conventions.

Important

For disabling output and stderr with --force-stdout-spec and --force-stderr-spec the values %NONE% and %NULL% achieve it, but with different effect. With %NONE%, the corresponding handle becomes None. As a result, e.g. sys.stdout will be None, which is different from %NULL% where it will be backed by a file pointing to os.devnull, i.e. you can write to it.

With %NONE%, you may get RuntimeError: lost sys.stdout in case it does get used; with %NULL% that never happens. However, some libraries handle this as input for their logging mechanism, and on Windows this is how you are compatible with pythonw.exe which is behaving like %NONE%.

Use Case 5 - Setuptools Wheels

If you have a setup.py, setup.cfg or pyproject.toml driven creation of wheels for your software in place, putting Nuitka to use is extremely easy.

Let's start with the most common setuptools approach, you can, having Nuitka installed of course, simply execute the target bdist_nuitka rather than the bdist_wheel. It takes all the options and allows you to specify some more, that are specific to Nuitka.

# For setup.py if you don't use other build systems:
setup(
   # Data files are to be handled by setuptools and not Nuitka
   package_data={"some_package": ["some_file.txt"]},
   ...,
   # This is to pass Nuitka options.
   command_options={
      'nuitka': {
         # boolean option, e.g. if you cared for C compilation commands
         '--show-scons': True,
         # options without value, e.g. enforce using Clang
         '--clang': None,
         # options with single values, e.g. enable a plugin of Nuitka
         '--enable-plugin': "pyside2",
         # options with several values, e.g. avoiding including modules
         '--nofollow-import-to' : ["*.tests", "*.distutils"],
      },
   },
)

# For setup.py with other build systems:
# The tuple nature of the arguments is required by the dark nature of
# "setuptools" and plugins to it, that insist on full compatibility,
# e.g. "setuptools_rust"

setup(
   # Data files are to be handled by setuptools and not Nuitka
   package_data={"some_package": ["some_file.txt"]},
   ...,
   # This is to pass Nuitka options.
   ...,
   command_options={
      'nuitka': {
         # boolean option, e.g. if you cared for C compilation commands
         '--show-scons': ("setup.py", True),
         # options without value, e.g. enforce using Clang
         '--clang': ("setup.py", None),
         # options with single values, e.g. enable a plugin of Nuitka
         '--enable-plugin': ("setup.py", "pyside2"),
         # options with several values, e.g. avoiding including modules
         '--nofollow-import-to' : ("setup.py", ["*.tests", "*.distutils"]),
      }
   },
)

If for some reason, you cannot or do not want to change the target, you can add this to your setup.py.

# For setup.py
setup(
   ...,
   build_with_nuitka=True
)

Note

To temporarily disable the compilation, you could the remove above line, or edit the value to False by or take its value from an environment variable if you so choose, e.g. bool(os.environ.get("USE_NUITKA", "True")). This is up to you.

Or you could put it in your setup.cfg

[metadata]
build_with_nuitka = True

And last, but not least, Nuitka also supports the new build meta, so when you have a pyproject.toml already, simple replace or add this value:

[build-system]
requires = ["setuptools>=42", "wheel", "nuitka", "toml"]
build-backend = "nuitka.distutils.Build"

# Data files are to be handled by setuptools and not Nuitka
[tool.setuptools.package-data]
some_package = ['data_file.txt']

[tool.nuitka]
# These are not recommended, but they make it obvious to have effect.

# boolean option, e.g. if you cared for C compilation commands, leading
# dashes are omitted
show-scons = true

# options with single values, e.g. enable a plugin of Nuitka
enable-plugin = pyside2

# options with several values, e.g. avoiding including modules, accepts
# list argument.
nofollow-import-to = ["*.tests", "*.distutils"]

Note

For the nuitka requirement above absolute paths like C:\Users\...\Nuitka will also work on Linux, use an absolute path with two leading slashes, e.g. //home/.../Nuitka.

Note

Whatever approach you take, data files in these wheels are not handled by Nuitka at all, but by setuptools. You can, however, use the data file embedding of Nuitka commercial. In that case, you actually would embed the files inside the extension module itself, and not as a file in the wheel.

Use Case 6 - Multidist

If you have multiple programs, that each should be executable, in the past you had to compile multiple times, and deploy all of these. With standalone mode, this, of course, meant that you were fairly wasteful, as sharing the folders could be done, but wasn't really supported by Nuitka.

Enter Multidist. There is an option --main that replaces or adds to the positional argument given. And it can be given multiple times. When given multiple times, Nuitka will create a binary that contains the code of all the programs given, but sharing modules used in them. They therefore do not have to be distributed multiple times.

Let's call the basename of the main path, and entry point. The names of these must, of course, be different. Then the created binary can execute either entry point, and will react to what sys.argv[0] appears to it. So if executed in the right way (with something like subprocess or OS API you can control this name), or by renaming or copying the binary, or symlinking to it, you can then achieve the miracle.

This allows to combine very different programs into one.

Note

This feature is still experimental. Use with care and report your findings should you encounter anything that is undesirable behavior

This mode works with standalone, onefile, and mere acceleration. It does not work with module mode.

Tweaks

Icons

For good looks, you may specify icons. On Windows, you can provide an icon file, a template executable, or a PNG file. All of these will work and may even be combined:

# These create binaries with icons on Windows
python -m nuitka --onefile --windows-icon-from-ico=your-icon.png program.py
python -m nuitka --onefile --windows-icon-from-ico=your-icon.ico program.py
python -m nuitka --onefile --windows-icon-template-exe=your-icon.ico program.py

# These create application bundles with icons on macOS
python -m nuitka --macos-create-app-bundle --macos-app-icon=your-icon.png program.py
python -m nuitka --macos-create-app-bundle --macos-app-icon=your-icon.icns program.py

Note

With Nuitka, you do not have to create platform-specific icons, but instead it will convert e.g. PNG, but also other formats on the fly during the build.

MacOS Entitlements

Entitlements for an macOS application bundle can be added with the option, --macos-app-protected-resource, all values are listed on this page from Apple

An example value would be --macos-app-protected-resource=NSMicrophoneUsageDescription:Microphone access for requesting access to a Microphone. After the colon, the descriptive text is to be given.

Note

Beware that in the likely case of using spaces in the description part, you need to quote it for your shell to get through to Nuitka and not be interpreted as Nuitka arguments.

Console Window

On Windows, the console is opened by programs unless you say so. Nuitka defaults to this, effectively being only good for terminal programs, or programs where the output is requested to be seen. There is a difference in pythonw.exe and python.exe along those lines. This is replicated in Nuitka with the option --disable-console. Nuitka recommends you to consider this in case you are using PySide6 e.g. and other GUI packages, e.g. wx, but it leaves the decision up to you. In case, you know your program is console application, just using --enable-console which will get rid of these kinds of outputs from Nuitka.

Note

The pythonw.exe is never good to be used with Nuitka, as you cannot see its output.

Splash screen

Splash screens are useful when program startup is slow. Onefile startup itself is not slow, but your program may be, and you cannot really know how fast the computer used will be, so it might be a good idea to have them. Luckily, with Nuitka, they are easy to add for Windows.

For the splash screen, you need to specify it as a PNG file, and then make sure to disable the splash screen when your program is ready, e.g. has completed the imports, prepared the window, connected to the database, and wants the splash screen to go away. 

from https://github.com/Nuitka/Nuitka