enqueue: CLI utility to queue command execution

Update: As you can see in the comments below, and as I feared, it turned out that Lluís Batlle i Rossell already implemented something much like Enqueue, only better in many regards. I doubt I’ll keep maintaining enqueue, there’s no reason to. Oh well, it was a nice afternoon project.

Something that always bugged me with my shell workflow is the problem of queueing commands to run one after the other, while adding commands to the queue as they previous commands are being executed.

Take, for example, a simple usecase: we want to move three large files from diskA to diskB. The problem is that you don’t know the name of the files in advance, perhaps because you’re renaming them manually and it takes you time to type, or because you’re hunting for them in the directory tree, or whatever. Here are some solutions to this:

• Start one command in the background, then do something like fg ; second-command. Then prepare the third command, but only run it after you saw the second finished. Meh. Or,
• Just let the jobs run concurrently in the background as you run them (using the & control operator). But since each command is maxing out a resource (CPU, disk, etc), this becomes woefully inefficient really quickly. Or,
• Use a mad concoction of Ctrl-Z, fg/bg, wait n or (if you left the shell and want to add something to the queue) use a madder concoction of while pgrep -f 'mv /disk/A/foo' > /dev/null; do ... (I’ll leave it as an exercise to the frustrated reader to finish that little one liner). But then again, you could also spend that time getting a paper-cut at the edge of your nostril, and it would probably be just as much fun and maybe even less error prone. Or,
• Start a shell process reading from a named pipe (mkfifo(1)), and write the commands to the named pipe (credit to my friend and colleague m0she for this sneaky idea). In practice, I found it unwieldy at best, and impossible to extend with bells-and-whistle features if you need them, first and foremost, easily listing the queue and your position in it.

I reckon you could think of a few more ways, but I doubt (and hope! 🙂 none would be more convenient than to simply use Enqueue, a simple and hopefully lightweight Python/twisted command line queuer (written today by yours truly). Usage looks a bit like so:
[sourcecode light=”1″]
$ alias nq=enqueue
$ nq add mv /disk1/file1 /disk2/file1
$ nq add mv /disk1/file2 /disk2/file2
$ nq add beep
$ nq list
* mv /disk1/file1 /disk2/file1
mv /disk1/file2 /disk2/file2
beep
$
[/sourcecode]
Nice and easy. Enqueue is still a bit rough around the edges and not very feature rich, but it does the job for me and I hope you’d like it too. Queues are managed by a twisted daemon that talks to the CLI client over UNIX domain sockets, and the whole thing fits in about 300 lines of Python. Feel free to open issues/send pull requests on GitHub if you find bugs or want to suggest something, I’ll try to keep up. Promise.

p.s.: Why is it that every time I dabble in Python packaging I end up (a) horribly frustrated and (b) feeling the result is awfully inadequate? Yes, I know I could probably package enqueue better, I know packaging isn’t Python’s strongest side and I know the future is better than the present, but the present sucks and I just had to say this. Yeah, I also walk around in the summertime sayin’ “how about this heat”.