It’s been over two years now since I started at the Wall Street Journal – back when I went from an amateur enthusiast programmer to a ‘professional’.
One of the most tangible results of these two years is that my code is cleaner and clearer than ever before. Most of my projects have been bespoke client-side apps with short shelf lives, which have provided opportunities to experiment and quickly learn what works (and what doesn’t!).
Here’s how the code I wrote for interactive graphics in early 2014 compares to the code I write now.
As my projects have grown in scope and ambition, so too has the corresponding code and, in turn, the difficulty of keeping it under control. Learning how to structure my code in a clean, logical and modifiable-on-a-tight-deadline manner has been a long process.
Then: I would typically have one big function called on
window.ready, which would itself contain smaller functions. There would still tend to be plenty of globals lying about the place, especially for configuration.
Now: I try to keep primary functionality in an object called
App, and then separate any substantial code related to specific functionality. For example, I typically move code for charts written with D3 – which tend to consist of at least several hundred lines – into constructor objects (a nice clean design pattern I picked up from my colleague Jason French).
Here’s an abstracted version:
Update: By popular request, I’ve since written a full blog post on using the D3 constructor pattern.
Next: We’ve yet to universally adopt a build system for interactive graphics at WSJ, but I’ve been experimenting with module-loading systems such as RequireJS and Webpack in my own time. They both allow one to split out JS code into smaller files and then
require them inline – as one can in PHP, Python or basically any other halfway-sane language – and encourages cleaner, self-contained code.
As a beginner, I assumed there’s only one way to write functions. They’re just blocks of code, innit? Over the past two years I’ve started to learn more specialised patterns.
Then: My functions would often serve many purposes at once, such as processing data (typically with huge and/or multiple
for loops) and appending HTML in one go.
Occasionally they would take long lists of arguments, which can then be confusing to modify later. For example, if I wanted provide a height when running
makeArc in this one case, which argument would I change?
height is actually controlled by a fifth argument, which is absent in this case.
Now: I try to give each function a clear purpose. Sometimes it can be a ‘pure function’, which takes an input and returns an output without modifying anything else (a good model for ‘data wrangling’ functions). Or a function might return a closure, which allows me to keep all related variables inside of the original function. Other times I will avoid writing functions altogether and rely on battle-tested libraries such as Moment, which handles date interpretation, querying, modification and formatting.
For functions that take more than a couple of arguments, I pass in objects, which make it far clearer what each argument is:
Then: I would keep frequently-changing data in JSON files (loaded in using
jQuery.getJSON), and historical and config data inline in script files. I’d often paste spreadsheets into Mr Data Converter to turn them into JSON format.
Now: I prefer to keep everything in JSON files, and use
jQuery.when to determine when each
jQuery.getJSON has finished. Instead of using Mr Data Converter, I often write a Node script to convert from CSV to JSON, which has the added benefit of moving data processing out of the browser.
Next: I’ve considered using the reporter-friendly ArchieML data format for projects with large amounts of copy, but I’ve found it a pain to set up with Google Docs and the syntax isn’t as straightforward as I’d like. I would love to be convinced it’s worth using. (Actually, in the process of writing this article, I just came across Quartz’s aml-gdoc-server, which could be the exactly what I need.)
Aka ‘data wrangling’, this is the bit of the code that turns an imported JSON or CSV file into a sane data format compatible with Highcharts, Mustache or whatever one’s code needs to work with.
Then: As mentioned earlier, I would process data using great big
for loops, pushing the results into a new array or object (and sometimes appending HTML on the fly!).
Here’s a particularly nasty example. With code like this, it’d be easy to introduce a correction-worthy mistake.
The data would sometimes be stored within HTML data attributes, rather than an in-memory JS object; which in retrospect is utter madness.
These are initially a little harder to get one’s head around, but eventually result in much clearer code than huge
for loops with tonnes of
if statements. Only once these operations are finished do I begin to start generating HTML.
Next: If I were to start using ES6, I could make use of ‘fat arrow functions’ in my anonymous functions (of which there are many when using said array methods). Fat arrow functions not only save a few characters, but also maintain the scope of their parent, which is very handy when using object-oriented programming involving
Then: In a 2014 blog post, I wrote that “Handlebars, a dynamic templating library, can be useful on larger projects”.
Much of the time, however, I would concatenate strings and use jQuery very liberally. Here’s a relatively benign chunk of concatenation code – but if it required an
if statement, this would get hairy fast:
Now: I’ve since learnt the error of my ways: concatenating more than a short string is a recipe for disaster and should be avoided at all costs. Instead, I use the Mustache library with abandon, introducing it early in the project before the temptation to start concatenating becomes too strong.
Here’s a Mustache template from Barrel Breakdown:
I’ve gotten into the habit of using Mustache over Handlebars because (a) it has a smaller page-weight, (b) the templates can be rendered server-side using the Mustache PHP library if necessary and (c) I don’t need most of Handlebars’ fancy features.
Now: For better or worse, I now worry less about page weight (much of which is ad scripts and web fonts anyway) and more about saving time and reducing the risk of bugs: hence, more liberal use of libraries. Moment.js and Mustache are essentials.
Next: If I were to use a module loader such as Webpack, it might make sense to use a package manager such as npm; but with so few dependencies for each project, it’s still more practical to keep them within the repository.
My early projects tended to be big on maps, whereas nowadays my projects tend to be focused around charts. My general approach has evolved from trying to do as much as possible with just HTML & CSS, to Highcharts (and sometimes awkward hacks) to actually getting to grips with D3.
Then: Mostly relying on HTML & CSS, with some slightly-wobbly D3. Looking back, it’s almost as if I was intentionally avoiding traditional bar and line charts – perhaps because I had no idea how to code them.
Now: A mix of tools: HTML & CSS for basic horizontal bars, Highcharts for simple charts and D3 for complex designs. Now that I feel comfortable with all three approaches, it’s just a matter of choosing the one most appropriate for the project.
Next: Even with D3’s useful scale, axis and line generators, I still end up rewriting similar code for each project in order to match our style guide. I’ve been thinking about writing some sort of D3 boilerplate for myself.
Then: When I joined WSJ, we had just taken out a CartoDB subscription. It had a slick interface and worked on mobile; what’s not to like? Let’s use it everywhere!
Now: After some struggles with the CartoDB JS API, I realised that it used Leaflet as a base – and so that library has since become my go-to when I’m in need of an interactive map (indeed, it’s the basis of Pinpoint). For choropleth maps, shapefiles rendered standalone using D3 often look better than on a CartoDB basemap.
Next: We’ve been making heavy use of ai2html over the past year, especially for maps. Unless they’re showing live results or a particularly dense dataset, most maps don’t really need to be interactive. Using a custom version of ai2html, our in-house cartographers can apply their existing expertise to produce maps far better-looking and easier to read than those made with Leaflet or otherwise.
Posted 09 May 2016