Giving it a restGiving it a rest

Last year Databinder interfaced with Ruby for text processing, using XML-RPC because it seemed like the easiest option. The project ran years late and millions of dollars over budget.

Just kidding—it went fine of course. (And Databinder nets a cool zero million no matter which way the wind blows.) Using a scripting language in an external runtime for text processing, and caching the results, has been an unmitigated success. This is why probably no one else in Java is doing it. They would rather use those what you see is the tip of an ugly iceberg editors and warehouse a bunch of disgusting HTML generated by Microsoft Word. (This happens by way of the clipboard.)

But! Smart people prefer REST over XML-RPC. It’s true. Anything that abstracts remote procedure calls over HTTP is bad according to them, even things less obviously torturous than SOAP. Maybe they’re right. It’s definitely true that you don’t need XML to represent the single input and output strings of this chore, nor the abstraction of remote procedure calls.

Some parts of the REST creed are unappealing. Basically everything starting with the word “always.” And Wicket as a web framework is the anti-REST. So, there’s that. But unlike actual religion, with programming religions you can pick the parts you like and use them without being burned at the stake. At least not physically.

The part about REST that rules is just getting stuff done with the world’s de facto standard of software communication whenever feasible. Databinder’s simple problem, shuttling markup strings between the JVM and Ruby runtime, can be directly handled by HTTP. To use any other protocol, or layer something on top of this one, is just silly.

Old databinder-dispatch was a Java library depending on some XML-RPC libs that themselves depended on HttpClient. New databinder-dispatch dumps those other libs and also (why not?) is written in Scala. Um,

Dispatch is the first Databinder module to be rewritten in Scala. Last June this weblog brashly pro­claimed its hope to rewrite the entire toolkit in Scala. Plans change though! Instead of porting the now-plural persistence modules, it makes more sense to factor out IModel using first class functions. A page would acquire needed values, some how, and bind them to components as by-name parameters.

val post_method = new PostMethod(server_base + path_name)
post_method.setParameter("input", source.toString())
 "Content-Type", "application/x-www-form-urlencoded; charset=utf-8")

(new HttpClient).executeMethod(post_method) match {
  case 200 => ()
  case code => error("Bad response code: " + code)
val out = post_method.getResponseBodyAsString()

So basically it’s posting some UTF-8 text to an endpoint defined externally. And on the Ruby side:

srv = :Port => 8180 )
def create_server(srv, name, &convert)
  srv.mount_proc('/' + name) do |req, resp|
    require name
    resp.body =['input'])
    resp['Content-Type'] = 'text/html; charset=utf-8'

The only annoying thing here is having to verbosely specify UTF-8 all over the place. But if you don’t do that, and then roll out to some environment that defaults to the evil Latin-1, forget about your em dashes and curved quotes surviving a round trip. The Ruby side doesn’t seem to pay much attention to the encoding either way, but HttpClient needs to know on both ends that it’s UTF-8 or it will assume whatever the JVM’s default is. (It would be nice if there were a terse way to set a local default when instantiating the HttpClient!) (It would be nice if WEBrick echoed whatever content type encoding header came in!)

Later: The trunk code is using HttpClient 4 inside a Scala wrapper (inherited, not implied), that changes the client’s default to UTF-8.

All told, the HTTP-only code was not any harder to write than the original XML-RPC code was. (For more extensive Scala REST coding it would be worth writing a few implicit wrappers for HttpClient.) The two can’t be compared line to line, since the Java became Scala and the Ruby has become a fuller-fledged init.d service, but feel free to poke around:

And of course there are some pre-fab components set up to use the HttpConverter that entirely abstract out the remote business. The old version had a base XmlRpcLabel and a pile of subclasses in separate source files because in Java you have no choice. The new version just has the HTTP converter and puts all the text format classes (which are mostly one-liners anyway) in a single TextFormattedLabel source file.

The top-level Scala components like TextileLabel can be as easily accessed from Java as regular Java classes. But working with the format signifiers—if you need to select a format at runtime—is trickier, because they are singleton objects like Textile$.MODULE$. But it does work.

Next week let’s rewrite this in SOAP for another comparison! Just kidding that would suck.


Dump Webrick, get Thin instead (gem install thin), then run:

thin start –stats /stats

Point the browser to localhost:3000/stats and you can see the headers of the last request.

Any particular reason to use an external Ruby instance rather than embedded JRuby? I would imagine this would be somewhat easier to deal with, especially since it doesn’t require an entirely separate app stack.

Assaf, that would have been helpful for debugging the character set. Although, what I meant is that if I post with a certain claimed encoding, I would like for the response to also claim that encoding. Maybe I will try in Thin and see if does.

Daniel, I think it will be easier some day but at the moment it probably sounds a lot easier than it is. I did the analogue once with Jython, and got it working eventually. The thing is, I already have stable Ruby environments on all my servers where I can do things like gem update maruku. My feeling is that JRuby is not there yet, especially not if I stick with embedded Jetty. (Glassfish, forget it.) Plus I don’t particularly want to burden my app instance with a scripting runtime when it’s already got scala-lib and such. So I find this approach to be counterintuitively simpler than the one that would occur to most people (and myself, with Jython) first.

HTTP makes a separation between the request format and response format. To tell the server which format you accept, you use Accept headers, the one you’re looking for is Accept-Charset.

That allows you to post url-encoded or multipart with a JPEG image, but get back an HTML document, or post something in Russian but get back a UTF-8 page that mixes multiple languages.

Thin won’t break that behavior.

I just tried Accept-Charset, but it doesn’t change the outcome; WEBbrick replies without specifying any content type. I suppose the problem is I’m expecting it to look at the response.body I’ve set (a string) and add the meta data. But it doesn’t know the character set, and couldn’t know what the text represents. So this is fine I guess.

It better not change the outcome. If Webrick started messing with the response body, a lot of apps would go breaking.


Hello, I would like to try something fresh and new like REST vs the old-way of doing a web app CRUD but right now I’m up-to-my-neck in web-app rewrite from someone that used a code-generation tool. Though, Hibernate is considered a code-generation tool the application I am dealing with was written completely front to back including the ugly 2005 SQL Server database, no, database is not correct. More correctly this code-generator created 1500+ tables of 2005 SQL Server repository of everything web-app including the storage of HTML snippets. The database finds no PK or FK anywhere! This is why I am attempting to use Databinder. I enjoyed an immediate modicum of success right away with the baseball web-app example but things quickly came to an end when I tried to use variables of type: Integer, Long or anything that was not a String. The real show-stopper is there is no support to be had anywhere.

geezenslaw, if you see this message, please check your spam folder or settings. I’ve replied to your e-mails and not gotten a bounce back. You have an account on the Databinder forum, the password is in the email. I’ll be happy to answer your questions on the forum.

Anybody knows usable xml-rpc library for scala?

Add a comment