Skip to end of metadata
Go to start of metadata
You are viewing an old version of this page. View the current version. Compare with Current  |   View Page History

Brief intro to DSLs

Fandoc says:


DSLs or Domain Specific Languages allow you to embed other languages into your Fantom source code. The syntax for a DSL is:

AnchorType <|...|>

Everything between the <| and |> tokens is considered source code of the DSL itself. The anchor type defines how to the compile the DSL. DslPlugins are registered on the anchor type, and called by the Fantom compiler to translate them into a Fantom expression.

Technically speaking, DSLs are presented as DSL expressions, and it's result type is defined by the anchor type, so that

  str := Str<|hello, world!|>  // str has type Str
  foo := Foo<|bar|> // foo has type Foo

Therefore, the most obvious way to use DSL expressions is instantiation of some complex objects. For example, imagine we are writing a library for graph manipulation, so we define classes like Graph, Node, and Edge. See
And imagine that we need to instantiate some graphs for our tests, so we write the code like this:

 a := Node("a")
 b := Node("b")
 c := Node("c")
 d := Node("d")
 ab := Edge(a, b)
 bc := Edge(b, c)
 bd := Edge(b, d)
 graph := Graph([a,b,c,d], [ab, bc, bd])

What a lot of code! Let's use some DSL magic:

  graph := Graph<|a -> b
                  b -> c
                  b -> d|>

The DSL plugin for Graph just parses the code between <| and |> (which is accessed via compiler::DslExpr.src) and generates appropriate object creation.

However even such a simple example is quite tricky - to allow our DSL to be used as field initializer, or default parameter value, we need to convert our DSL code to a single expression.

So, before implementing the DSL plugin itself, we need to understand how we can replace our code with a single expression. In this particular example, assuming that Node and Edge classes override equals and hash correctly, it can be done like this:

  ["a","b","c","d"].map { 
    ["b", "c"], 
    ["b", "d"]].map { 

The source code of DSL plugin can be found here.

Uh, after looking at the source of GraphDsl, the question is - why do we want to write DSL plugins? The same task can be fairly easy implemented in simple static method like Graph.fromStr. Why anyone want to use heavy low-level Compiler API? The benefit like compile-time validation and generation of compile error on a bad line seems to be too small, almost negligible.
That's what I thought when saw Fantom DSLs for a first time, and then forgot about them almost for a year.

Beyond the DSL src

Last week, being tired from code like this:

    if(node is ListType) return selectType(node->valType)
    else if(node is TypeDef) return selectTypeDef(node)
    else if(node is CType) return selectType(node)
    else if(node is SlotRef) return selectSlotRef(node)
    else if(node is SlotDef) return selectSlotDef(path)
    else if(node is MethodVarRef) return selectMethodVarRef(node)

I thought it'd be cool to have multiple dispatch in Fantom. The initial implementation supposed to be quite simple - we have Dispatcher class which looks like this:

const class Dispatcher
  new make(Func[] funcs := Func[,]) { this.funcs = funcs }  

  public Dispatcher add(Func f) { Dispatcher(funcs.dup.add(f)) }
  ** Dispatch can accept more args than needed for a func,
  ** so funcs in this container can have different arity
  Obj? call(Obj?[] args) {
    //dummy approach for now - 
    //find first function which accepts less than arg count 
    //arguments with all param types fitting to arg types
    (funcs.exclude |f|
      args.any |arg, i| 
        f.params.size > i && 
        !f.params[i].type.fits(arg?.typeof ?: Obj?#) 
    }.first ?: throw ArgErr("No matching functions for args $args")).callList(args)

Using this, the code above can be written like this:

    d := Dispatcher([
      ... ])[this, node])

Slightly better, but still a lot of boilerplate code. What if we'd use DSLs + Symbols here? Using that, we can write code like this:

class SelectionEngine
  @Dispatch Void selectTypeDef(TypeDef t) 
  @Dispatch Void selectType(CType t)
  @Dispatch Void selectSlotRef(SlotRef s)
  Void select(Node node) { Dispatcher<|select|>.call([this, node]) }
  //everything dispatched according to node type, 
  //unsupported node types immediately give us exceptions 

So, all we need to do is to write simple marker facet:

facet class Dispatch {}

And DSL plugin which will make everything for us - find all methods annotated with @Dispatch and starting with a given prefix, take functions from them and then pass list of functions to Dispatcher constructor. Sounds fairly easy, but when I started implementing it, I found the first problem - from a DslPlugin.compile we don't know where we are - I mean, we don't know anything about enclosing type or method.

Luckily for me, DslPlugin extends CompilerSupport, which means we have full access to all compilation units and type definitions for them! So we can take location of our DSL expression, and then by iterating through all compilation units and comparing location, we can find our compilation unit. Using the same way, we can find enclosing type definition.
The rest is simple - iterate through all slots of our type, find all methods with given prefix and facet and construct expression for Dispatcher creation.

Nice! But wait for a second - this means that new instance of Dispatcher will be created per each method invocation. That's not exactly what we want. What if we could inject a private static const Dispatcher selectDispatcher? And yes, we can! So, right inside our DSL plugin, we can write something like this:

    fieldName := "${name}Dispatcher"
    field := FieldDef.make(loc, parent, fieldName, 
      FConst.Const + FConst.Private + FConst.Storage + FConst.Static)
    field.fieldType = ns.resolveType("mdispatch::Dispatcher")
    field.init = //Expression to create dispatcher object

However, there's one more thing we have to do - modify static initializer for our type. Static initializer is a special method generated by compiler and it contains all assignments made to static field definitions:

class Foo
  static const Str str := "a"
  static const Int int := 45
  //this method is generated by compiler
  private static synthetic Void static$init()
    str = "a"
    int = 45

At the stage when DSL plugins are called, this static method is already generated (if there are other static fields in this class), so we need to manually find this method and modify it's code (if there are no static fields, we also need to create this method):

    staticInit := type.methodDef("static\$init")
    //insert statement at the end of the list
    staticInit.code.stmts.insert(-1, BinaryExpr.makeAssign(fieldRef(dispField, loc), initExpr).toStmt)

And voila! Everything compiles and runs smoothly now. Here's the code. However it already smells like black magic. Let's go further!

Static imports

Not so long ago there's a Static imports discussion on Fantom's forum. Without arguing whether static imports are good or bad, let's see what we can do using DSLs, so the code like this would be possible:

class Sample
  StaticImports imports := 
  Void main()
    file := os("/")
    prefix := spaces(4)    

So, what we want here - the DSL source just defines a list of fully qualified static methods, and DSL plugin just creates local static methods which route to corresponding 'foreign' static methods:

class Sample
  private static File os(Str str) { File.os(str) }

The creation of method routers and their bodies look fairly simple, but there's a problem - again, DSL plugins are called a bit too late and compiler have already generated a lot of compiler errors. Fail? No! We can do another black magic trick - go through all complier errs and remove those which caused by (yet) undefined methods:

//build qnames of new instance methods
newQnames := { "${type.qname}.${it[it.indexr(".")..-1]}" }
//build compiler error message for further filtering
errMsgs := { "Unknown method '$it'" }
compiler.errs = compiler.errs.exclude { errMsgs.contains(it.msg) }

However we also need to manually go through all expressions and adjust CallExpr instances to insert correct references to our newly created methods. After then everything compiles and runs as expected, though console output will still display compilation errors during building. The dirty code of StaticImportDsl can be found here.

In fact, the example above have nothing common with initial purpose of DSLs, it is complete compiler plugin which significantly changes the source code meaning.

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.