How destructuring saved the Empire

As a longtime Ruby programmer who has just recently begun to focus more on Clojure, I find myself continually surprised by all the little treasures that Clojure offers. One of those treasures is Clojure's sophisticated support for destructuring - it's even better here than in EcmaScript 6! To explain the beauty of Clojure's support for destructuring function arguments, we need a good example.

Let's pretend that you are the lead Engineer for the Empire (yes, the one from Star Wars). The Empire, having conquered the entire galaxy, needs to keep tabs on a vast number of inhabited planets. Besides the run-of-the-mill monitoring needed to keep the Imperial Infrastructure up and running for your loyal citizens, you also need to know when it's time to unleash a little of that Dark Side to counteract threats. Sometimes, a surprise visit from a Sith Lord will be good enough, but when things get really bad (and - let's be honest - they get really bad a lot), you need to decide which planets to destroy with the Death Star.

Due to a sequence of...unfortunate mishaps, you've learned the hard way that you need to automate this observation. Human Lackeys just aren't reliable enough. Being an evil genius, you've obviously set up Riemann to help you process the incoming events. An observation satellite has been dispatched to each habitable planet, in order to report the midi-chlorian levels, the rebel scum head count, and the tithes generated by each planet. At the Imperial Capitol, the Riemann server receives these events and decides what to do. Here are some sample events:

{:host "Onderon" :service "Midi-chlorian Levels" :time 1455741552 :state "ok" :metric 23 }
{:host "Bespin" :service "Rebel Scum" :time 1455741552 :state "ok" :metric 392 }
{:host "Corellia" :service "Imperial Tithe Credits"  :time 1455741552 :state "ok" :metric 92220011 }

If a planet starts to emit dangerously high midi-chlorian levels, we know that a Jedi problem is likely to emerge soon. And if the Rebels are running thick, sometimes the simplest solution (total destruction) is the easiest. But, either of these factors can be ignored for a while if the planet's tithe proves significant... it's a balancing act.

Since we need to know all three of these readings in order to determine whether to vaporize a planet, in our main Riemann stream, we "project" the three events to a single function. This function in our riemann-config.clj will blow the planet up with the Death Star, if it remains too threatening to justify the income it provides for a whole day:

(by :host
  (project [(service "Midi-chlorian Level")
            (service "Rebel Scum")
            (service "Imperial Tithe Credits")]
               (smap (partial consider-deathstar settings)
                     (stable (24 * 60 * 60 * 1000) :state
                             (where (state "critical")
                                    (fn [event]
                                      (log/warn "Blowing it the #*$(@$#* up.")
                                      (blow-up-with-deathstar (:host event))))))))

Here's a naive way to write a Riemann fold function that operates on this trio of events, and, based on the metrics, decides whether a planet seems to worthy of being allowed to continue to exist.

(defn fold-for-deathstar
  "Compares planetary metrics against configured Imperial thresholds;
if the planet's threat outweighs its benefit, will return an event
in critical state!"
  [settings events]
  (event
   :service "deathstar consideration"
   :state (if (or (> (/ (:metric (first events)) (:metric (last events)))
                     (:midi-chlorians-tolerance settings))
                  (> (/ (:metric (second events)) (:metric (last events)))
                     (:rebel-tolerance settings)))
            "critical" "ok")
   :description "Based on these readings, do we need to keep this planet around?"
   :metric nil
   :tags nil))

There's a real problem with this function. It is very difficult to tell that the function is correct by reading it (and in the Empire, you need everyone to know that you are correct because the consequences of failure tend towards the immediate and dire). Your only way to discern what the second parameter "events" is supposed to contain is by fully analyzing the algorithm - and even then, you have no clue about the meaning of the individual events. Because it encourages communication to lapse between the architect of this function and the team that uses it, this type of function is highly likely to result in planets accidentally being left intact.

If confusion is the problem, is adding documentation the solution? Take 2:

(defn fold-for-deathstar
  "Compares planetary metrics against configured Imperial thresholds;
if the planet's threat outweighs its benefit, will return an event
in critical state!
  Parameters:
  settings: A Map containing threshold levels we care about:
    :midi-chlorians-tolerance and :rebel-tolerance.
  events: should be a collection consisting of three Riemann events structures, in this order:
    Midi-chlorian count, Rebel Scum count, and Imperial Tithe credits."
  [settings events]
  (event
   :service "deathstar consideration"
   :state (if (or (> (/ (:metric (first events)) (:metric (last events)))
                     (:midi-chlorians-tolerance settings))
                  (> (/ (:metric (second events)) (:metric (last events)))
                     (:rebel-tolerance settings)))
            "critical" "ok")
   :description "Based on these readings, do we need to keep this planet around?"
   :metric nil
   :tags nil))

This is a little better, but it's always problematic to ask people to check something other than the code itself in order to decipher the meaning. For one thing, it's an inescapable reality that most fresh recruits to the Imperial Engineering Team don't "read the manual". For another thing, code comments are notorious for quickly becoming inaccurate. Someone changes the function, but forgets to update the documentation, and no one notices, because, for the time being at least, everything continues to work!

Most importantly, even if you do absorb the documentation, and it stays correct, verifying the correctness of this function requires repeatedly cross-referencing what you see in the code below with its description in plain English way at the top. There would be less mental effort involved if the more of the meaning were contained in the terms of the code itself.

So, how about this instead:

(defn fold-for-deathstar
  "Compares planetary metrics against configured Imperial thresholds;
  if the planet's threat outweighs its benefit, will return an event
  in critical state!
  Parameters:
  settings: A Map containing threshold levels we care about:
    :midi-chlorians-tolerance and :rebel-tolerance.
  events: should be a collection consisting of three Riemann events structures, in this order:
    Midi-chlorian count, Rebel Scum count, and Imperial Tithe credits."
  [settings events]
  (let [midi-chlorians (:metric (first events))
        rebel-scum (:metric (second events))
        tithe (:metric (last events))
        midi-chlorians-tolerance (:midi-chlorians-tolerance settings)
        rebel-tolerance (:rebel-tolerance settings)]
    (event
     :service "deathstar consideration"
     :state (if (or (> (/ midi-chlorians tithe) midi-chlorians-tolerance)
                    (> (/ rebel-scum tithe) rebel-tolerance))
              "critical" "ok")
     :description "Based on these readings, do we need to keep this planet around?"
     :metric nil
     :tags nil)))

We're getting there! Now that we've named some variables, the "meat" of the Death Star evaluation algorithm is very easy to understand at a glance.

Unfortunately, this clarity comes with a cost. Declaring the local variables has added a new section to the function, making it a little longer and more verbose. Clojure functions, in particular, tend to lose a little of their elegance from the introduction of a set of local variables. The magical conciseness of functional code has diminished somewhat, and we've actually had to add a new level of indentation to all of the "business code" in the function!

Here's where destructuring comes to the rescue. "Destructuring" is a form of local variable assignment that allows you give names only to the values you need from a piece of structured data; the data as a whole (here, "settings" and "events") can remain anonymous. Here's what our function can look like if we utilize Clojure's excellent support for destructuring arguments to functions:

(defn fold-for-deathstar
  "Compares planetary metrics against configured Imperial thresholds;
  if the planet's threat outweighs its benefit, will return an event
  in critical state!
  Parameters:
  settings: A Map containing threshold levels we care about:
    :midi-chlorians-tolerance and :rebel-tolerance.
  events: should be a collection consisting of three Riemann events structures, in this order:
    Midi-chlorian count, Rebel Scum count, and Imperial Tithe credits."
  [{:keys [midi-chlorians-tolerance rebel-tolerance]}
   [{midi-chlorians :metric} {rebel-scum :metric} {tithe :metric}]]
  (event
   :service "deathstar consideration"
   :state (if (or (> (/ midi-chlorians tithe) midi-chlorians-tolerance)
                  (> (/ rebel-scum tithe) rebel-tolerance))
            "critical" "ok")
   :description "Based on these readings, do we need to keep this planet around?"))

Destructuring has allowed us a chance to create something that approaches "living documentation", an opportunity which you should seize whenever it pops up. The elements we care about in the data structures passed into this function are now formally a part of the function's signature!

Now, even if someone doesn't read the documentation, it will be difficult to misinterpret how to use this function. In fact, I would argue that at this point, the text documentation about the parameters has become redundant and that we'd, therefore, be better off without it. Thus, we arrive here:

(defn fold-for-deathstar
  "Compares planetary metrics against configured Imperial thresholds;
  if the planet's threat outweighs its benefit, will return an event
  in critical state!"
  [{:keys [midi-chlorians-tolerance rebel-tolerance]}
   [{midi-chlorians :metric} {rebel-scum :metric} {tithe :metric}]]
  (event
   :service "deathstar consideration"
   :state (if (or (> (/ midi-chlorians tithe) midi-chlorians-tolerance)
                  (> (/ rebel-scum tithe) rebel-tolerance))
            "critical" "ok")
   :description "Based on these readings, do we need to keep this planet around?"))

Stand back for a minute and admire your handiwork! Those Jedis and rebels won't stand a chance.

It's perfect...

For now.