'Ducers Wild -- a concise guide to the menagerie

TL;DR

It's not too long, but, to summarize the summary, if you read Rich Hickey's 2014 blog post on transducers first, his 2012 post on reducers will be easier to understand.

Brief Definitions

Herewith, all in one place, are Clojuresque definitions of:

  • reducible
  • reducing function
  • transducer
  • reducer
  • folder
  • decomplected

Longer elaborations of these definitions follow in the subsequent section.

reducing function

Anything that can be used as the first argument of the reduce function, e.g. + or conj. Generally, it's a binary function that returns something of the type of its first argument, which is supposed ...

more ...

Testing on Hundreds of AWS Instances - (sort of Distributed functional programming, part 3a)

In part 1 and part2 of this almost unbearably exciting series, I outlined the concept of distributing purely functional programs and went through some implementation details of Girder.

So far, however, I've only asserted that it works, so today I want to start explaining how I used Amazon Web Services to test pretty cheaply on gobs of machines.

The art of AWS wrangling was somewhat new to me, and I went through a few iterations before deciding on the right mix of tools. Needless to say, the right mix ended up being fairly Clojure heavy, which helps to smooth ...

more ...

Comedy - A brief syllabus in honor of Joan Rivers

I have a confession to make. I don't find comedy funny. There are exceptions - Gilbert Gottfried and Louis CK can occasionally reduce me to putty - but, by and large, I appreciate comedy from a technical rather than aesthetic standpoint, combined with an admiration for and empathy with the drive that compels people to perfect their craft.

I have similar feelings about magicians. Having spent a significant portion of my youth reading about and reverse-engineering illusions, before concluding that certain qualities necessary for success in the field are innate rather than learned, and I would always lack them. For example ...

more ...

Distributed purely functional programming, part 2

Update 2015-01-12

The algorithm as it exists in HEAD is somewhat different from the below, in ways that I'll describe (eventually) in an another post. In some ways, it's closer to Fork-Join, but with important differences to support reentrancy, share results of duplicate requests and adjust for the costs of distribution.

Recap of recap

In a previous post, I introduced a framework called Girder (the code is on github), which aims to facilitate Plain Old Functional Programming on distributed systems. By POFP, I mean code that, as much as possible, consists of normal looking calls to normal looking ...

more ...

Quick tip on getting lein to recognize your new dependency

Ever realized you needed to modify some dependency (e.g. I needed to add weak cache support to core.cache), go a head with forking on github, lein install your new version, modify your project.clj to pick it up, and then... somehow... you still seem to get the old version? That's usually because some other dependency is explicitly requesting a version of that library. If you do lein classpath, you'll see your library show up, in two different versions, with the one you don't want first.

In my case, the relevant bit of classpath shows

   /Users ...
more ...

Beauty and the Beast - Distributed functional programming, part 1

Update 2015-01-12

The algorithm as it exists in HEAD is somewhat different from the below, in ways that I'll describe (eventually) in an another post. In some ways, it's closer to Fork-Join, but with important differences to support reentrancy, share results of duplicate requests and adjust for the costs of distribution.

OSS and Commercial Grids

Grid computing has always suffered the reputation of a buzzword that one suspects might not actually mean anything, but it has become especially ill-defined with the rise of open-source distributed computation frameworks like Spark, Storm and Grampa Hadoop. These extensively documented systems don ...

more ...

Yeah, yeah, I should have used perl

Overkill

After reading my compelling post on a clojure Slack-bot, an astute reader1 pointed out that using the fullblown apparatus of compojure, jetty, jvm, etc. for something this silly is really only justified when the entire purpose of the exercise is procrastination. Well, dear astute reader, that was the entire purpose, but you're right. Ok, here it is in perl,

    #!/usr/bin/env perl
    use Mojolicious::Lite;
    use File::Slurp;
    my $token = read_file("TOKEN");
    my @lines = split(/\0/,read_file("LINES"));
    post '/slacks' => sub {
      my $c   = shift;
      my $t   = $c->param('token');
      if ($t eq $token) {
          $c->render(json ...
more ...

Procrastination with a clojure M-x yow slackbot

Procrastination

I really should be working on this obscure distributed RDF/FRP thing, but for various reasons my head isn't working properly write now. So I did this other stupid thing instead.

Zippy

Once upon a time, M-x yow in emacs would deliver a nice random quote from Zippy the Pinhead. Nowadays, you just get

Yow!  Legally-imposed CULTURE-reduction is CABBAGE-BRAINED!

which has something to do with copyright law. More specifically, the file yow.lines in emacs' data-directory now contains only the opinion expressed above, rather than the original seven-hundred or so precious epigrams, delimited by \000. I have heard ...

more ...

Boyhood vs Little House Smackdown

She was glad that the cozy house, and Pa and Ma and the firelight and the music, were now. They could not be forgotten, she thought, because now is now. It can never be a long time ago.

Or, put another way, "It's constant, the moments, it's just - it's like it's always right now, you know?"

So little Laura figures it all out at the age of 5, while Mason comes to the realization only after implanting an ear gauge, leaving for college and getting high in a location of staggering natural beauty. But it's ...

more ...

Fortran, Clojure, Haskell and Julia are not at war

TL;DR

  • There was this perplexing article in Ars Technica about replacing Fortran for scientific computation.
  • The main performance advantage of Fortran is that its restrictive data structures facilitate vector optimization.
  • There are many ways to match and even outperform Fortran, while using other languages.
  • But it's OK to use Fortran if you want to.
  • Ars Technica notwithstanding, Clojure, Haskell and Julia are not locked in competition to replace Fortran.
  • Ars Technica notwithstanding, Fibonacci sequences may be something we learned about once, but it is difficult to disguise their irrelevance to scientific computing.

Preamble

Ten or fifteen pages into ...

more ...