Reflection (by the example of Python)¶

Accessing the AST¶

It is also possible to access code that is written in Python. This works using the ast module, and works as follows:

In [8]:
SRC = """
def f(x, y):
    return 2*x + y**2 + 5
"""

import ast
tree = ast.parse(SRC)

print(ast.dump(tree))
Module(body=[FunctionDef(name='f', args=arguments(args=[arg(arg='x', annotation=None), arg(arg='y', annotation=None)], vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), body=[Return(value=BinOp(left=BinOp(left=BinOp(left=Num(n=2), op=Mult(), right=Name(id='x', ctx=Load())), op=Add(), right=BinOp(left=Name(id='y', ctx=Load()), op=Pow(), right=Num(n=2))), op=Add(), right=Num(n=5)))], decorator_list=[], returns=None)])

It is possible to transcribe the expressions here into the form discussed earlier.

In [5]:
print(ast.dump(tree.body[0].body[0].value))
BinOp(left=BinOp(left=BinOp(left=Num(n=2), op=Mult(), right=Name(id='x', ctx=Load())), op=Add(), right=BinOp(left=Name(id='y', ctx=Load()), op=Pow(), right=Num(n=2))), op=Add(), right=Num(n=5))
In [6]:
from pymbolic.interop.ast import ASTToPymbolic
expr = ASTToPymbolic()(tree.body[0].body[0].value)
print(expr)
2*x + y**2 + 5

But beware when defining languages this way. Python has very well-defined semantics, and the user will expect that your way of executing their code is a good match for their mental model of what the code should do. As such, it may be better to start with a "blank slate" in terms of language design, so as to not run afoul of already formed expectations.

Accessing the Bytecode¶

In [7]:
def f(x, y):
    return 2*x + y**2 + 5

import dis
dis.dis(f)
  2           0 LOAD_CONST               1 (2)
              2 LOAD_FAST                0 (x)
              4 BINARY_MULTIPLY
              6 LOAD_FAST                1 (y)
              8 LOAD_CONST               1 (2)
             10 BINARY_POWER
             12 BINARY_ADD
             14 LOAD_CONST               2 (5)
             16 BINARY_ADD
             18 RETURN_VALUE

Observations:

  • Stack-based machine
  • Operations corresponding to Python's abstract object model
  • Notice anything about local variables?
  • See Hy for a lisp-y language implemented on the same VM. Demo