Build your own builderBuild your own builder

If you’ve ever coded using the Wicket framework, you know there is almost no overhead in it to create a custom component with an anonymous subclass. This practice is so normal that some of the standard components are abstract classes. And because you are always extending components for one-off customizations, it’s easy to factor these into your own reusable component classes. A natural progression occurs from anonymous classes to nested, then static, and finally top-level classes. That is, for custom components that are broadly applicable. Those that remain one-off, or useful only within a particular scope, stop evolving at the appropriate stage.

The design of the Scala builder simple-build-tool similarly anticipates extension of its project traits and classes. When invoked in a project, sbt looks for a top-level concrete project class to process. Any behavior overrides in that class are executed when compiling and packaging the project. But you’re not limited to patching the default build process; entirely new tasks defined in the project become available in sbt’s interactive mode and help screens, making even the user interface of the build environment extensible.

Building blocks for building

Most of sbt’s actual functionality exists in the traits and classes in its project hierarchy, which you are free to combine and use or not use. This makes your project definition—your own software—a peer of the builder itself. Aside from its launcher and the environment in which it invokes your project, the builder is effectively a library that you’re using to to write your own project builder.

Sounds like overkill, right? It would seem that most projects don’t need to implement their own builder, just like most webapp pages should not need to customize components. But once you have the ability, it turns out that you use it. Writing code is what we do, after all. The alternative—using a library that treats customization as a special case—is like trading a fishing pole for a fish. So a project built by sbt that may have started out relying on the default build definition tends to gain tasks for things that would have ended up in external scripts (or not automated at all) with a less easily extensible builder.

But no sooner do you write some useful building code than you want to reuse it in other projects. Fortunately, sbt supports reuse through plug-in modules. These are, really, just a way to add software to the builder’s classpath. That software can be project traits and base classes, or it could be a cookie recipe; sbt does not care as long as it’s bytecode or raw Scala.

So a project’s runtime classpath includes three categories of software peers: your project source, any referenced plug-ins, and sbt itself. Just by using the builder you’re potentially writing a future plug-in, or prototyping something that could find its way into the sbt core. Once again, it’s a natural progression from getting today’s work done to helping yourself and others in the future. The environment is highly productive both for the programmer and the sbt ecosystem as a whole.

Processing with spde-sbt

Once upon a time there was a Processing Development Environment fork. It was called Spde, a play on the PDE abbreviation that is pronounced “speed” in order to confuse you. The original PDE is—bear with me here—is like a dragonfly in amber. It’s small, fast, beautiful, and relatively frozen in time. It’s a minimalistic labor of love in a class of software (IDEs) that is dominated by awkward behemoths relentlessly bulked up by armies of programmers. The feature-sets of those IDEs have started to resemble a surrealist prank. But on the other hand, they still do things that users want and that the PDE’s embedded version of jEdit can not approximate.

The main problem for Spde however had nothing to do with Eclipse envy, but rather the limited potential of coding for the PDE’s walled garden. The forked environment, which we can now call Spde Classic, has for example a very handy ability to export Proguard-shrunk applets. Any applet using Scala practically needs this, otherwise it forces the client browser to download several megabytes of Scala jars. These JAR optimizers are generally no fun to configure, but in the end it worked. The only problem was that this broadly applicable software was bound up in an expressly limited environment.

It’s hard enough to be motivated to write free software, but to write it for the currently tiny intersection of Scala and Processing programming proved too difficult. Especially when a much better option became available a half year later in the form of sbt plug-ins. And so spde-sbt was born and that is essentially what Spde is today: a collection of sbt project traits and classes that facilitate coding in Scala against the standard Processing libraries.

Aside from spde-sbt’s potentially larger audience, it’s also nice just to be outside of an IDE again, even a tasteful one like the PDE. Now if you want to create your Proguard-compressed applet on a remote terminal instead in a desktop application, you can do that. You can have automated builds. You can customize builds. You’ve got the powah.

The most basic feature of the Spde plug-in, residing in the SpdeProject, is a transitive dependency on the Spde and Processing libraries. A client project that implements this trait doesn’t have to concern itself with the corresponding library versions. Beyond that the trait supports source wrapping similar to that done by Spde Classic; it combines the *.spde files in the project base directory into a “glob” proxy drawing class. (The proxy is necessary to avoid running the class initialization code too early.) This process produces a standard Scala source file under src_managed that is added to sbt and handled normally from that point on.

There are also traits to add OpenGL, and video playback through GStreamer. Projects need only extend the traits containing features they want to use. To show how this all fits together, the spde-examples project on GitHub uses the different project traits and base classes in subprojects. Old favorites like Flocking appear there, as well as some mathy new contributions from the folks at EPFL.

The only downside is that Spde’s new environment is not so easy to use as Spde Classic. To be productive with sbt you need to have command line know-how, and perhaps that is not a strength of all computer artists. So there is a Spde Graft that gives you some icons to double-click on, but not a lot of encouragement or feedback in general. Improving this situation will be interesting work, to say the least! But whatever comes of it will be useful across the sbt community.

Also, here is a video for anyone that doesn’t like the new Spde.

Codercomments

mmmmm… cookies…

Trending is an awesome example.

-Mark

Thanks! Trending was the pièce de résistance for my Spde talk at ITP, and I think it did succeed in elevating a few heads.

Prince55ing… ;)

Add a comment