Runaway processingRunaway processing

Once upon a time there was a graphical programming princess named Processing. She conjured the most beautiful animations and interactive art in all the world! Every artist wanted to work with her. But the princess’s castle lay within a vast colony of millworkers that, excepting her own court, had little appreciation for her talents. Indeed, she was better known outside the colony than inside. The few millworkers that did request an audience with the princess required her to wear their tool-laden overalls instead of her own elegant and simple gown. As time passed, the exotic artists and artisans grew weary the journey to the mill, and to avoid this displeasure they they contrived a replica of the princess in papier-mâché. Meanwhile, a promising new land was being cultivated…

Retrofitting the Processing Development Environment for Scala

Processing is both a language and environment. While in this case language refers also to the API, Processing has a distinct syntax known internally as p5 that is converted into Java by a preprocessor. This syntax dispenses with some qualities of Java that may be irksome to new programmers (or old programmers) with the help of an ANTLR generated parser.

Where the specialized syntax has benefits, it also has costs. The build is more complicated, as ANTLR must first run against the grammar to generate Java sources to parse p5. This process is automated by shell scripts. Additionally, the number of people that are comfortable working on language grammars is small; the p5 grammar has yet to be updated to Java 1.5 generics and loops.

The Processing language and environment are being held back, presently, by the depth of their original ambition. Writing a customized Java grammar was the right thing to do at the time of Processing’s conception, with the JVM as the best cross-platform virtual machine and Java as its only fully realized language. But now only the first of those things is true. Basing Processing on the standard version of a more modern language would free it from a complicated build process and the need for regular attention from language wizards.

The language proposed here for that job is Scala, which ships off boatloads of Java cruft on sails of type inference. Beyond that it allows API designers to vacuum remaining code hairballs with implicit conversions, first-class functions, and object mix-ins. All of the syntax substitutions made by p5’s preprocessor are either unnecessary in Scala or can be equivalently performed by an underlying interface and implicit conversions. One pioneer has blazed this trail to the point of calling the core library from Scala, but bringing the language into the environment is the key to preserving—and even improving—the remarkable student-friendliness of Processing.

Showing, not telling

Once you get into Git, you become a fork terrorist. Ohh ho ho… what will we fork next? Web frameworks with inadequate support for RFC 2616 section 13? Probably! But this week it was the Processing Development Environment.

The fork started off as an attempt to replace the Eclipse Compiler for Java, used by the PDE, with Scala’s. But then one thing lead to another and both the ECJ and ANTLR dependencies were on the cutting room floor. The result:

Explode, in Scala

Explode running

That’s a translation of a basic Processing example. The only structural change is the dispersal of setup code. In p5’s active mode, operations like size must be performed inside a method like setup. But in Scala the wrapped top-level code becomes the base constructor, which is nested here in a draw proxy that is constructed in SApplet’s setup method.

This conceit frees the user from needing to know the difference between construction and setting up, and allows global values to be initialized using methods that are not otherwise available until after construction (eliminating a few lines of code in this case). It’s half-way to the simplicity of Processing’s static mode and could eliminate the need for it, particularly since you can replace the blank template for new files with one that defines a blank draw method.

After the fork

For the moment the base SApplet class is still in Java, and lives in a forked core module. If this experiment becomes anything serious it would be smarter to work with a completely stock core and build a separate module for Scala extensions, written in Scala. The forked development environment will need a distinct name (Spde?) and branding graphics (???). That all sounds really time consuming, but also fun, and not that difficult since a lot of old stuff can be discarded. Instead of running sketches in a separate process it might be faster to do it directly and with some live reloading potential thanks to Scala’s interpreter-concious compiler and/or JavaRebel’s free Scala license.

Hello! Now the fork has been converted to Buildr and is in a fresh, tiny repository. The original clone will be used to merge upstream changes but won’t be published.
Hello again! Now Spde has a site with downloads so you can skip all this building business.

Does anyone want to help with this? Is itself interested in moving to Scala? Whatever works! Probably what wouldn’t work so well is for a lot of people to try to clone this branch from, which is technically on a DSL connection. Many jars, and old versions of jars, are embedded therein. Once the fork is converted to Buildr (oh, yes) those can be burned from the history (or maybe it is easier to just have github serve the ancient jars and everything else). But if you really want to play with it now and make improvements then you can try a shallow clone (see note):

git clone git://

And as for JavaScript: this ain’t over yet.


Wow! Good demonstration of what Scala is capable of in terms of domain-specific languages. The critical point now is to be able to integrate the expressiveness of Processing with standard Java/Scala. That may be something shown in the example you gave, but I haven’t had a chance to clone the repo yet (crazy iPhone still doesn’t have git).

Anyway, I personally quite like working on grammars in languages like ANTLR, but a scala DSL is certainly easier to adapt and integrate. Nice work!

Thanks! A Scala API on top of core will be awesome. Just need to rework the build process to support it.

I’d love to help, as I’ve been mixing scala and processing via HipsterIncs method for a while now.

I know, I linked! Would love to see any code you’ve been working on since that post. Hm I just looked at github and rediscovered that the free account is limited to 100MB. I’ll get the blobs out of my branch as soon as I can so it will be easier to share this.

Cool, I’ll clone at home… (that sounds weird)

Great posts! I managed to adapt your Flocking code to work with Scala and Processing in Eclipse, and then was able to export it as a single jar with the Fat Jar plugin, but the file size included the 3.5mb for scala-library.jar.

I then tried to use ProGuard on my own, but it kept failing, I think on this line: Warning: flocking.Flock$$anonfun$1: can't find referenced field 'processing.core.PApplet flocking$Flock$$parent' in class flocking.Flock Do you have the options file for ProGuard that SPDE uses? Any other thoughts?

zip file is here

ah, bad href for the zip file, try this

Hi Steven, the ProGuard config is in Sketch. Also SApplet may be of interest. The way Spde works currently is, the sketch is inserted as a DrawProxy implementation within SApplet. There are just a few things to change if you’re embedding directly in a PApplet, which I think you have discovered.

Thanks for the response! And sorry it took me so long, it’s been a busy week.

I did a Clean in Eclipse, and that seems to have resolved that specific ProGuard warning - it now completes, but when I try to launch the jar, it gives an “Exception in thread “main” java.lang.NoSuchMethodError: main”

I’m explicitly keeping processing.core.PApplet though, and I thought main was in that?

(I’m going to reply on Steven’s blog.)

Add a comment