I’m building a site with Databinder that’s almost entirely images (much unlike databinder.net, which is all about text). Every bit of copy on it has been hand-drawn, scanned, and processed. Not by me, obviously.

When I saw how the site was going to work I suggested to its designer that she think seriously about hiring some person, some other person, to type up fifty or so static HTML pages, put PayPal purchase links on each, and call it a day. But the client wanted a proper shopping cart, so we decided to do it up. There was no way I was going to manage those images by hand, so I would have to make a backend for the designer to upload them, and set all the other data. Off we went! (This was, like, two weeks ago. I’m very sleepy.)

Insoluble conflict

I’ve fought with my share of mutable, filesystem-based Web images. And they won. I’ve made a Wicket page for “editing” them that fails in all sorts of unfortunate ways that I avoid looking in to. Even if that page were flawless, there would still be a five minute delay between uploads and availability (yes, rsync running on a cron job), which I see as failure in iteslf. Why bother polishing a manure shovel?

Compared to the using the filesystem, storing images (or other binary data) in databases is a dream. We all know this. You just ask for the image associated with the record you’re already working with, and it’s yours. Null it, replace it, whatever, and you don’t even have to have permission! Databases have learned to chug all those nameless bits just fine, and Hibernate doesn’t drop the ball either. Wicket practically turns cartwheels to get you to do database images.

There’s just one problem: serving images from the database is slow as all hell, for obvious reasons. Unless you’ve got money to burn on server hardware, you can’t serve a large volume of images from the database (via Hibernate via Wicket). You may as well send postcards.

Funny how the more that you need database images (because you have fifteen kinds of images associated with various datatypes), the more you can’t spare the processing time to pipe them. But wait—here comes Mr. Deus ex Machina! (God, he has saved my ass so many times.)

Mr. Machina

If you place your Web application behind a “reverse” caching server, you can eliminate the performance drag (completely) and keep the benefits of database storage (completely). Whoa. The cache will even be faster than a Web server pointing to the filesystem. Win, win.

Well, okay, it’s never that easy. I spent more hours than I counted fighting with the Varnish HTTP accelerator, but in the end, I won. My setup rules. It even invalidates (via HTTP purge) cached images when you upload new ones, all from one reusable Panel. For now I’m only using Varnish for images, but setting cacheable headers on static Wicket pages will be an easy performance gain when I get around it to it.

Varnish does interfere a bit with some of Wicket’s tricks (like the way it caches redirects, but you can work around that). One thing I couldn’t work around was, ironically, Varnish stalling out on multipart form posts for the very images I was using it to accelerate. My solution (after assuming it was a deadlock in MySQL, Hibernate, or Wicket and wasting several hours looking for it) was to leave the uncached Jetty port open for site administration.

What, like you never cheat?