When creating DSLs from a programming languages, there are two ways of implementing it: Internal and External.

Implementing an External DSL implies creating the code for all the steps:

Lexer -> Parser -> ASTTranslator (Frontend) -> Interpreter/Output/Bytecode (Backend)

An internal (embedded) DSL means reusing parts of the programming language interpreter/compiler:

* Lexer/Parser: Adding new recognized token
* AST: Modifying the methods to generate the AST
* Backend: Modifying the output generated from the AST

See this blog post of Fernando Meyer’s blog. He modifies python Parser to recognize new words as tokens, and maps them to existing tokens.

I am wondering is there is a way to provide an useful and easy to use abstraction to modify python AST generation. Pydsl is designed to work with external DSLs, but there are obvious advantages on implementing internal DSLs. Although I think python is not the best language for this purpose.

For more info see Martin Fowler post about DSLs.