The PPCI (Pure Python Compiler Infrastructure) project is a compiler written entirely in the Python programming language. It contains front-ends for various programming languages as well as machine code generation functionality. With this library you can generate (working!) machine code using Python (and thus very easy to explore, extend, etc.)!
Mython is an extensible variant of the Python programming language. Mython makes Python extensible by adding two things: parametric quotation statement, and compile-time metaprogramming. The parametric quote statement is simply syntactic sugar for saying “run some function on this embedded string”. Compile-time metaprogramming allows you to evaluate that function on the embedded string at compile time. This gives you added choice, both in terms of what your code looks like, and when you want to evaluate that code.
➜ Not sure what this really means, and if it’s useful (it was never released)
➜ There are a few research papers.
- https://dl.acm.org/doi/10.1145/1837513.1640141 “Language embedding and optimization in mython”
Interesting techniques / libraries¶
- https://github.com/sdiehl/pyrewrite.git (fetch)
- https://github.com/true-grue/raddsl.git (fetch)
- https://github.com/sdiehl/subpy (fetch)
- https://github.com/chrisbouchard/termination.git (fetch)
“Improving performance of Python code using rewriting rules technique”
http://ceur-ws.org/Vol-2866/ceur_115-125jereb11.pdf / http://pp.isofts.kiev.ua/ojs1/article/view/403
“We propose an approach to increasing performance of Python code by transforming fragments of code to more efficient languages such as Cython and C++. We use high-level algebraic models and rewriting rules technique for semi-automated code transformation. Performance-critical fragments of code are transformed into a low-level syntax model using Python parser”
- ‘astpretty’: Pretty print the output of python stdlib
- ‘astunparse’: An AST unparser for Python. This is a factored out version of unparse found in the Python source distribution; under Demo/parser in Python 2 and under Tools/parser in Python 3.
- astor (previously codegen): Python AST read/write
- Asteria (
Al2O3): Missing AST features (monkey-patches ast using astor / astpretty):
- gast: A generic AST to represent Python2 and Python3’s Abstract Syntax Tree(AST). GAST provides a compatibility layer between the AST of various Python versions, as produced by ast.parse from the standard ast module.
- beniget: A static analyzer for Python2 and Python3 code. Beniget provides a static over-approximation of the global and local definitions inside Python Module/Class/Function. It can also compute def-use chains from each definition.
Parsers (for other things)¶
- ‘numexpr’: The
numexprpackage supplies routines for the fast evaluation of array expressions elementwise by using a vector-based virtual machine.
Books and tutorials¶
Course/Book: Essentials of Compilation - An Incremental Approach in Python
https://github.com/IUCompilerCourse/Essentials-of-Compilation + https://github.com/IUCompilerCourse/python-student-support-code
➜ Interesting tutorial / book for undergraduate students. Compiles Python to machine language. Easy to read.
“Let’s Write an LLVM Specializer for Python!”
http://dev.stephendiehl.com/numpile/ and https://github.com/sdiehl/numpile
➜ Very interesing tutorial. But old.
➜ Hints at some additional ideas (including using https://github.com/sdiehl/pyrewrite and https://github.com/sdiehl/subpy)
➜ Shows some basic compilation techniques (Python AST ➜ Python Bytecode)
➜ Repo here: https://github.com/darius/tailbiter
https://arxiv.org/abs/2011.13127 “Copy-and-Patch Compilation: A fast compilation algorithm for high-level languages and bytecode”
➜ Maybe interesting (or not) for us. At least, contains references to recent papers.