Parallelize all the things -- Deconstructing Haxl, with Clojure macros, topological sorting and Quasar fibers

Parallel Lines

Select, few

Recently, there's been a lot of interest in DSLs to address the so-called N+1 selects problem, which describes the very common situation where you

  1. Do one database query to get a list of things, say user IDs;
  2. for each, call some sort of processing function that happens to
  3. do another query, say for each user's name.

It can be even worse than that, as in this example, where (for some reason) we want to append the last name of every person in a group to the first name of their best friend. (Maybe they're getting married.)

(defn …
more ...

Losing my religion: Switching to windows + linux under Hyper-V

(See Updates section at end for latest notes.)


In the middle of the journey of our life I came to myself within a dark wood where I was no longer comfortable identifying myself as a Mac user. Superficially, the relationship with my MacBook air had been rewarding. It lasted all day, ran the necessary few bits of commercial software I needed, and otherwise did a great job of pretending to be running unix. Also, I felt a pleasant sense of well-moderated individuality when my fellow developers and I exchanged knowing glances at our discreetly weathered, FP-inflected decal collections.

But there …

more ...

Scala Partial Functions Are Disgusting and Unnecessary

Thou whoreson zed! thou unnecessary letter!

- King Lear, 2.2.61

Partial functions are great in theory. By "great in theory", I mean that the basic idea is pretty simple but with very little additional research you can make weighty pronouncements involving the words "in" and "theory", as in, "in category theory, something something surjective algebra manifold."

The simplest example of a partial function is something like the logarithm, which isn't defined for arguments less than or equal to zero. Of course, it's also not defined for arguments that are strings or colors or egg-laying mammals, a restriction we usually …

more ...

Know when to fold 'em: visualizing the streaming, concurrent reduce

Holy cow! Can you believe your luck? What better way to spend some portion of whatever day of the week it is than to think about parallelizing reduction algorithms?

Really? You can't think of one? In that case, I'll kick off our inevitable friendship by contributing a clojure implementation of a fancy algorithm that asynchronously reduces arbitrary input streams while preserving order. Then I'll scare you with glimpses of my obsessive quest to visualize what this algorithm is actually doing; if you hang around, I'll show you what I eventually cobbled together with clojurescript and reagent. (Hint. It's an ANIMATION …

more ...

A minimalist translation of Clojure's core.async to Scala


Great computer languages, like monarchs and planets, become emblems of what surrounds them. The greatest computer languages are barely there, as nearly everything we file under their names could be described as a library or other customization. It's not unusual and not even absurd to find a question about socket() on a C language forum: Linux is arguably a morphogenic implication of C. Clojure, too, is formally minimal, but it induces composition. Much of what we specifically admire about it isn't the language itself so much as an expression of it.

Clojure provides Communicating Sequential Processes (CSP) via the …

more ...

Look at the Pie. Or my new, crappy watch I made myself.

Mutt: What will be the appropriate designation for the upcoming period of solar rotation in the Anno Domini system as devised by St. Dionysius Exiguus?

Jeff: Why don't you just look at the pie?

Mutt: Couldn't you just answer the question? Why do you always have to be a sarcastic asshole?

Jeff: Seriously, look at the pie.

Mutt: Huh. Yeah. I see what you mean. Sorry dude.

Jeff: No problem, my tinhorn peer. It is after all New Years.

Ens.: And to all a good night.


Consciously uncouple from the yoke of technological precision, using nothing but precision …

more ...

Squiggly Lines: Background lint- and type-checking for Clojure in emacs.

In the alternate universe where I am very honest, my résumé contains a line item for the aggregate year I've spent fiddling with init.el. It does not, however, list emacs lisp among the languages I know, because (in this alternate universe) I have too much self respect to flaunt cut and paste skills.

In the present universe, this is going to be awkward, because I will have to present really crappy elisp code without apology.

What this is about.

  1. One of the things you get in an industrial strength IDE, in exchange for giving up your favorite editor, is …
more ...

NYC Clojure Meetup slides on lenses and appropriate typing

To everyone who attended yesterday's NYC Clojure Meetup: thanks for listening to me, asking good questions and providing some pretty great answers as well.

Here are the slides. For more detail on nearly everything, see previous posts.

(Navigate using the compass arrows. Up/Down within a section; Left/Right betwen sections; ESC for overview.)

more ...

Lost in Transduction - Heresy and ingratitude edition

Were you at Clojure/conj in Washington last week? If so, hello again. Wasn't that a great conference? If not, head to Clojure TV, where all the talks are ready for streaming. Assuming some moderate level of Clojure obsession on your part, I couldn't recommend skipping any of them, so the full catch-up might take you a while, but there are two in particular that I strongly recommend.

Avoiding altercations

The first is actually the very last talk of the conference. Brian Goetz, whom you may have encountered previously as the author of Java Concurrency in Practice or heard of …

more ...

Stateless but state-aware types for transducers in Scala, using what seems to be magic

How strange to think that, a mere week ago, the world had not yet heard my public pronouncement that transducers ought to be stateless. True, the media frenzy has died down a bit, but in its stead comes the quiet awareness that life will never be the same. Anyway, that's the way I like to think about it.


  1. Storing state in the transducer makes it mutable, which might be unfortunate on general principles. In any event, it interferes with the metaphor of transducers as recipes for transformation and arguably makes code more difficult to understand.
  2. A natural place …
more ...