Technically speaking, DSLs are presented as DSL expressions, and it's result type is defined by the anchor type, so that
Therefore, the most obvious way to use DSL expressions is instantiation of some complex objects. For example, imagine we are writing a library for graph manipulation, so we define classes like Graph, Node, and Edge. See Graphs.fan
And imagine that we need to instantiate some graphs for our tests, so we write the code like this:
What a lot of code! Let's use some DSL magic:
The DSL plugin for Graph just parses the code between <| and |> (which is accessed via compiler::DslExpr.src) and generates appropriate object creation.
However even such a simple example is quite tricky - to allow our DSL to be used as field initializer, or default parameter value, we need to convert our DSL code to a single expression.
So, before implementing the DSL plugin itself, we need to understand how we can replace our code with a single expression. In this particular example, assuming that Node and Edge classes override equals and hash correctly, it can be done like this:
The source code of DSL plugin can be found here.
Uh, after looking at the source of GraphDsl, the question is - why do we want to write DSL plugins? The same task can be fairly easy implemented in simple static method like Graph.fromStr. Why anyone want to use heavy low-level Compiler API? The benefit like compile-time validation and generation of compile error on a bad line seems to be too small, almost negligible.
That's what I thought when saw Fantom DSLs for a first time, and then forgot about them almost for a year.
Last week, being tired from code like this:
I thought it'd be cool to have multiple dispatch in Fantom. The initial implementation supposed to be quite simple - we have Dispatcher class which looks like this:
Using this, the code above can be written like this:
Slightly better, but still a lot of boilerplate code. What if we'd use DSLs + Symbols here? Using that, we can write code like this:
So, all we need to do is to write simple marker facet:
And DSL plugin which will make everything for us - find all methods annotated with @Dispatch and starting with a given prefix, take functions from them and then pass list of functions to Dispatcher constructor. Sounds fairly easy, but when I started implementing it, I found the first problem - from a DslPlugin.compile we don't know where we are - I mean, we don't know anything about enclosing type or method.
Luckily for me, DslPlugin extends CompilerSupport, which means we have full access to all compilation units and type definitions for them! So we can take location of our DSL expression, and then by iterating through all compilation units and comparing location, we can find our compilation unit. Using the same way, we can find enclosing type definition.
The rest is simple - iterate through all slots of our type, find all methods with given prefix and facet and construct expression for Dispatcher creation.
Nice! But wait for a second - this means that new instance of Dispatcher will be created per each method invocation. That's not exactly what we want. What if we could inject a private static const Dispatcher selectDispatcher? And yes, we can! So, right inside our DSL plugin, we can write something like this:
However, there's one more thing we have to do - modify static initializer for our type. Static initializer is a special method generated by compiler and it contains all assignments made to static field definitions:
At the stage when DSL plugins are called, this static method is already generated (if there are other static fields in this class), so we need to manually find this method and modify it's code (if there are no static fields, we also need to create this method):
And voila! Everything compiles and runs smoothly now. Here's the code. However it already smells like black magic. Let's go further!
Not so long ago there's a Static imports discussion on Fantom's forum. Without arguing whether static imports are good or bad, let's see what we can do using DSLs, so the code like this would be possible:
So, what we want here - the DSL source just defines a list of fully qualified static methods, and DSL plugin just creates local static methods which route to corresponding 'foreign' static methods:
The creation of method routers and their bodies look fairly simple, but there's a problem - again, DSL plugins are called a bit too late and compiler have already generated a lot of compiler errors. Fail? No! We can do another black magic trick - go through all complier errs and remove those which caused by (yet) undefined methods:
However we also need to manually go through all expressions and adjust CallExpr instances to insert correct references to our newly created methods. After then everything compiles and runs as expected, though console output will still display compilation errors during building. The dirty code of StaticImportDsl can be found here.
In fact, the example above have nothing common with initial purpose of DSLs, it is complete compiler plugin which significantly changes the source code meaning.
So, using DSL plugins it is possible:
- Create objects
- Modify types by adding new methods
- Modify method bodies
- Remove compilation errors
But also, we can extend existing types! So, inside DSL plugin we can create new type extending anchor type, and then the instance of our new type will be just upcasted to the base type. On of the examples where it can be useful, is the following:
Here the parser grammar defines a set of non-terminal symbols which can refer to other symbols. We can generate the class extending 'Parser' with a set of methods corresponding to symbols right at compile time, something like that:
And therefore, since our parser extends Parser, the code like this will work correctly and give us great performance:
I'm not a parser guy, and don't have a complete working example on my hands, but I verified that extending types is possible on a simple synthetic example:
The Foo DSL plugin creates class like this:
DSL plugins is the backdoor which gives a full access to compiler API and allows to do almost everything. However, in such use cases, we don't need the return value of DSL expression. One of the ways of 'organizing chaos' can be something like below.
Let's define facet Magic:
It's has a single purpose - we can put it on type or slot. The field kind has a single purpose too - we can assign DSL expression to it.
Also, we can define the base class for our 'magic' plugins, which could simplify the creation of plugins. Here's the basic idea:
The implementation of type and slot methods searching for appropriate definitions can be found here.
Later, we can write plugins as simple like this:
And use them like this:
The DSL plugins seem to be powerful and dangerous, though quite complex tool, allowing to completely change the meaning of the source code. They strongly rely on Compiler API, which is quite big and likely to be changed. Also there's no clear understanding at which stage the DSL plugins are invoked, and sometimes it may be hard to predict all consequences of AST modifications, so it might be too risky to write something heavy using DSLs.
On the other hand, in some cases it can provide pretty elegant and robust solutions which can replace a ton of boilerplate code with full support of compile-time validation.
So, it seems to me that DSLs today are sort of black magic, and probably one day it should be revisited and well-organized.