Libjit

Libjit is probably the best free tool for realization of a quick, dynamical compilation. Libjit is used for realization of domain-specific programming and for acceleration of dynamic languages, such as Python or Lua.

The Libjit library provides a standard means for just-in-time compilation and quick compilation. Libjit, unlike many libraries and tools (such as VCode, NanoJIT, GNU lightning, GCC, and LLVM), provides a faster compilation time, is much more friendly for programmers, and is compatible with POSIX. Libjit is a more reliable means of compilation, and support is easier. In addition, all its source code and the high level interface have been well thought out and described in documentation.
Libjit is a solution for many operating systems and hardware components, for example GNU/Linux, GNU, Hurd, Linux, Debian, Ubuntu, Suse, Red Hat, Fedora Core, Windows, Windows Vista, Windows XP, and many other operating systems and hardware and software platforms.
This Web page contains the documentation and source code of the “Libjit-linear-scan”. This is a library which is based on the source code of Libjit and uses its API. For instance, studies and development of Libjit were conducted in various areas of dynamic compilation, including algorithms generating machine code and allocating registers.
The area of research includes the algorithms of allocation of registers with linear scanning, ECMA-334/ECMA-335, and Microsoft Common Intermediate Language, which are specialized for embedded systems and for computations in the cloud.
If you are looking for the official version please check out the DotGNU site.

Free software for advanced just-in-time compilation and dynamic compilation

Libjit has two goals. Firstly, it is a standardized API. Secondly, it has open source code for any modifications. Rhys Weatherley and Norbert Bollow created the design of the Libjit library and the base of its source code. The originally formulated goal was a library for dynamic compilation for the free software community that can allow it to compete more or less equally with different closed utilities for dynamic compilation using C language. GNU Portable.NET is an extensive set of tools which build a Common Language Infrastructure and which were started by the Free Software Foundation and follow all its fundamental principles.
Later Kirill Kononenko, Klaus Treichel, Aleksey Demakov, and other independent researchers took the source code of Libjit and made progress in the development of Libjit. These researchers examined its source code and compared it with its concurrents. They developed new research directions based on Libjit. Moreover, together they built a new dynamic compiler which is optimized for the GNU Portable.NET, based on the concept of parsing Common Intermediate Language. As a result they released the official package of Libjit 0.1.2. This version of Libjit is used for an efficient dynamic optimization in the software of industrial automation and in the work of financial institutions.

FAQ

1. Is it a good idea to use Libjit in my project?
2. Why should I use dynamic compilation with Libjit and API of Libjit?
3. Why does this Web site exist? Is this a fork of Libjit?
4. Should I use this version of Libjit?
5. How and why was this source code built?
6. Where is the Libjit library used?
7. Where can I find the documentation and the tutorials?
8. What mailing lists and Google groups are there?
9. How can I contribute to the development of Libjit?
10. How can we obtain some improvements or functionalities that we would like to have in the source code?
11. What other projects are related?

1. Is it a good idea to use Libjit in my project?

2. Why should I use dynamic compilation with Libjit and API of Libjit?

We think so. You are freed from the need to “reinvent the wheel” by building your own means of dynamic compilation starting from the scratch
Firstly, Libjit supports a standartized API. Secondly, Libjit is independent of the format of source bytecode. Thirdly, Libjit is independent of the garbage collector, shared libraries of the operating system, and other specifics of the virtual machine, which are related to the operating system. In addition, it provides a standard means for implementation of arithmetic operations, implementation of types conversions, handling of programs exceptions, and management of memory.
Unlike VCode, NanoJIT, GNU lightning, GCC, and LLVM, Libjit provides many nice features. For example:

• Compilation with Libjit aims to be deterministic and to have a linear-time.
• Libjit is suitable for "just-in-time" compilation and fast dynamic compilation.
• The binary code that Libjit generates is of very high quality for a quick compiler.
• Libjit is a library which is platform-independent. Minimal support for an execution of a byte-code, requires a compiler of C language, for example GNU Compiler Collection, Visual Studio or an IBM compiler. Support of an aggressive, optimized, and hand-crafted generation of machine code for target platforms includes:
• IA-32 (this support is of high quality which is suitable for production)
• AMD64 (in a major stage of development)
• ARM (we have experimental support)
• Alpha DEC (implementation was started by Thomas Cort in Google Summer of Code 2006)

We believe that the support of new hardware and software platforms and their testing may be achieved by the efforts of new software developers and engineers very quickly, in the shortest time. When Libjit “does not have the means” to generate a hand-crafted machine code, it “falls back” into a mode of interpretation of an intermediate form of the program.

• Its portable Applications Programming Interface has been tested very well. For example, it was used in embedded systems based on Linux and Windows, which work in real-time. It was applied in several industrial software applications, in the product automation industry, and in the machinery industry.
• Libjit has a remarkably easy API for software engineers. A simple compiler can be built in five minutes. Simple compilers, like VMKit project has done for MSIL and Java, can be developed in a couple of weeks as a course project. Compilers with a higher level of support at the level of industrial production can be developed in a timescale of two to three months.
• Libjit is a free software product and is independent of any proprietary interests.
• Libjit is a product that is free. Hence, Libjit can be used freely by any commercial software and in all scientific research. Or it can be applied in any open source community, by any free software community, or in closed software communities.
• Libjit can be applied in runtimes which want to implement an execution of MSIL, Java bytecode, Ruby, Python, Perl, LLVM (LLVA) bytecode, and many other virtual machines.
• Libjit allows quick compilation to be supported in a software project only once, and then it is used for a very long time. Software products which use Libjit can use new hardware features such as the multi-cores without any extra changes.
• Libjit is a project which is open to new ideas and new functionalities. Hence, it can be applied in industrial applications and scientific research projects.
• The team of engineers and the researchers who are involved in the development of Libjit are hard-working and open-minded and have a high level of cooperation.

3. Why does this Web site exist? Is this a fork of Libjit?

4. Should I use this version of Libjit?

- This is not a fork of Libjit. This is a research direction, based on the official Libjit version.

5. How and why was this source code built?

When an integration of a variant of an optimal linear time register allocation algorithm was made in Libjit, it was found that we need a lot of more research and development to make the optimization useful. We created this branch of Libjit. The support of a few optimizations was added. For example, we support various "optimal" register allocation algorithms, optimization based on data-flow, and control-flow analysis. These algorithms can be enabled with various optimization levels. What is more, they are adaptive. It is hoped that this research will help others: not just us, but also different software projects that already use or plan to implement dynamic compilation.
You can look at this review and the downloads section to find answers to many questions including a short review about how productivity with Libjit design and its approach for just-in-time compilation and fast dynamic compilation compares with solving this task using LLVM or GNU lightning.

The old source code of the Libjit-linear-scan 0.1.2.5 is available for download from the repository on this Web site.

The old source code of “Libjit-linear-scan 0.1.2.5” can be found in the SVN repository. You may look in the “downloads” for documentation, the research papers, benchmarks of performance, and the review of architecture.
In this release a few new features of the official repository have also been included:

• Levels of optimization for IA-32 from 0 to 4 (jit_function_set_optimization_level)
• A new kind of specialized Applications Binary Interface (jit_abi_internal)
• A brand new generator of binary code which supports four levels of optimization
• Several low-level optimizations
• Aggressive optimizations of algorithms that perform division of integers using multiplication (for details see Torbjorn Granlund and Peter L. Montgomery in “Division by Invariant Integers Using Multiplication”, divcnst-pldi94.pdf)
• Templates for generation of MMX/Streaming SIMD Extensions/SSE/SSE2/SSE3 instructions
• Usage for operations with floating point values SIMD SSE/SSE2/SSE3
• Usage for finding live variables at level 2 of optimization of a static analysis, which works fast
• At level 4 of optimization, elimination of source code which is found to be dead
• At levels 3 and 4 of optimization, usage of a static analysis to determine all live variables
• At level 2 of optimization, usage of a linear scan algorithm which is based on the register allocation algorithm by Massimiliano Poletto and Vivek Sarkar
• At levels 3 and 4 of optimization, usage of allocation of registers with second-chance binpacking
• The design and architecture were tested, and this extension was used for .NET real-world applications that run with GNU Portable.NET

6. Where is Libjit library used? For example, do you use it?

Below are presented a couple of examples in which Libjit was used:

• Libjit was applied to software of embedded systems in industrial lasers which produce the TRUMPF Laser. Those lasers make use of the tools of GNU Portable.NET and its managed implementation of Windows.Forms.

• Libjit research is applied as a research base in a few independent research projects:

• ILDJIT, which is developed by Simone Campanoni and the Formal Languages and Compilers Group Politecnico di Milano. See ILDJIT for the documentation and papers about how Libjit has been applied to the ILDJIT.

• Another application of the Libjit library research is in HornetsEye, which is developed by Jan Wedekind and the Mobile Machines and Vision Laboratory (MMVL), which is a part of the Centre for Robotics and Automation (CENRA) at the Materials and Engineering Research Institute (MERI). See the HornetsEye Web site for documentation, some papers, and videos about how Libjit is used for processing in real-time of image data and videos with Ruby.

7. Where can I find the documentation and the tutorials?

Libjit documentation.
The primary interface is in C, for maximal reusability. Class interfaces are available for programmers who prefer C++.
Designed for portability to all major 32-bit and 64-bit platforms.
Simple three-address API for library users, but opaque enough that other representations can be used inside the library in future without affecting existing users. Up-front or on-demand compilation of any function.
In-built support to re-compile functions with greater optimization, automatically redirecting previous callers to the new version.
Fallback interpreter for running code on platforms that do not yet have a native code generator. This reduces the requirement for the programmers to write their own interpreters for these platforms.
Arithmetic, bitwise, conversion, and comparison operators for 8-bit, 16-bit, 32-bit, or 64-bit integer types, and 32-bit, 64-bit, or longer floating point types. Includes overflow detecting arithmetic for integer types.
Large set of mathematical and trigonometric operations (sqrt, sin, cos, min, abs, etc.) for inlining floating-point library functions.
Simplified type layout and exception handling mechanisms, upon which a variety of different object models can be built.
Support for nested functions, able to access their parent’s local variables (for implementing Pascal-style languages).
-Rhys Weatherley

8. What mailing lists and Google groups are there?

• Libjit
• Libjit-linear-scan
• GNU Portable.NET

If you would like to know about usage of Libjit with other projects, then you could contact the mail lists of those projects. We believe the following projects may be interested in implementation of dynamic compilation using API of Libjit:

• Mono
• GCC
• Perl
• Python
• Ruby

9. How can I contribute to the development of Libjit?

10. How can we obtain some improvements or functionalities that we would like to have in the source code?

Your offers of work and monetary donations are welcome so that we can save more of our precious time.
Please use GNU savannah and the mailing list of Libjit to submit your patch or to report bugs which you find. Please do not hesitate to tell us your thoughts and share your ideas about how you want to use Libjit.
Please let us know your suggestions about how the Libjit community can improve Libjit even more for you, your business needs, or your research project or course work.
If you are interested in submitting your own patch, want to submit a new "smart" algorithm, or have any other need for a repository access, then you will probably want to have access to the repository of Libjit-linear-scan or Libjit.
If you would like to send your source code or if you plan to participate in development then please contact us so that we can cooperate with the highest level of productivity.

11. Related projects

Mono is a project led by Novell (formerly by Ximian) to create an ECMA standard compliant, .NET-compatible set of tools, including among others a C# compiler and a CLR implementation

.NET Framework includes a large library of coded solutions to prevent common programming problems and a virtual machine that manages the execution of programs written specifically for the framework. The .NET Framework is a key Microsoft offering and is intended to be used by most new applications created for the Windows platform.

.NET Micro Framework is a .NET platform for extremely resource-constrained devices

Cosmos is an extensible embedded operating system written in C#. The operating system is compiled together with a user program and associated libraries (written in any .NET language) using IL2CPU to create a bootable standalone native binary. The binary can be run off a floppy, usb key, or inside a VM. Cosmos is in early stages of development. The currently supported architecture is x86.

SharpOS is an open source .NET-C# based operating system currently under development by a group of volunteers and presided over by a team of six project administrators (Mircea-Cristian Racasan, Bruce Markham, Johann MacDonagh, Sander van Rossen, Jae Hyun and William Lahti).

Hornets Eye is a Ruby real-time computer vision extension running under GNU+Linux and Microsoft Windows

The Ludicrous JIT Compiler.

from http://code.google.com/p/libjit-linear-scan-register-allocator/