A 176 second job behind a 100 second timeout

The problem

Convert an arbitrary ebook or PDF into the format a 2011 Kindle Touch will actually render well, over HTTP, from a phone.

The constraint

Two numbers decided everything. The machine has 954 MiB of memory. The edge in front of it cuts a response off at roughly 100 seconds.

A single conversion of a 2.8 MiB file takes about 176 seconds. So the obvious design, where you post a file and get one back, is not merely slow. It cannot work at all.

The decision

Three things follow directly from those numbers.

A single-slot queue with ten waiting jobs, so only one conversion ever runs. Two concurrent conversions on 954 MiB is an out-of-memory kill, and a queue that admits unlimited work is just a slower way to fall over.

Memory cgroups on the unit, so a pathological input that makes the converter balloon gets killed before it takes the host and every other service on it down with it.

A poll-based protocol: submit, check status, download. No client ever holds a request open long enough to be severed, and the job survives the client going away entirely. Jobs and their outputs live on disk with an expiry, so a restart does not lose work in flight.

The result

Live, and it has never needed a bigger machine. The queue depth of one looks like an admission of weakness and is actually the whole reason it stays up.