2020 primary election results

There's a saying in news nerd circles: elections are nothing but edge cases. It's not that the data is inherently complicated — it's not — but US elections are a federated system, meaning that we actually have 50 smaller elections, and each of those is run by extremely messy humans. All of this is especially true of the primaries. It can't help but be complicated.

My team didn't fully realize, when we started 2020, how much work we would do on primary election results. NPR had not covered primaries at this level of detail before, and I don't think the editors on the politics beat really understood the scale of work involved (they probably still don't). Covering a general election is bad enough — covering elections over a nine-month period, especially as a pandemic wreaks havoc on the calendar, is an entirely different level of complication.

If you're interested in how the overall primary rig works, you can read our retrospective and check out the source code (full disclosure, you'll need an AP elections API key to actually run it). For this particular example, I want to drill down a little bit, into the county detail display. You can see one of those in action here — specifically, the state map and county results table at the bottom of the page.

I built most of the initial displays for the primary results, and set up the basic architecture for the application, but by the time Super Tuesday rolled around it was a team effort from myself, Audrey Carlsen, Alyson Hurt, and Ruth Talbot. Looking through the pull request commit log, we can see that Audrey did most of the work on the map itself and its integration into the bigger display.

Most of our primary results displays were built as pairs of elements, with an outer component that loaded and monitored data files for updates, and then passed those to the inner component for rendering. This architecture let us re-use results tables and other UI code across lots of different kinds of races, from house primaries to special general elections. We're just going to look at the <county-detail> component, which coordinated data between two child components.

_template.html

As you'll see in the template below, which is used to set the light DOM of the component, <county-detail> creates child components for display, and coordinates data between them: <county-map> for geographic display, <results-table> for county-level listings, and a drop-down UI menu to flip between counties.

<county-map data-as="map" aria-hidden="true"></county-map>

<div class="county-results" data-as="resultsContainer">

  <div class="controls">
    <label for="county-select">Select a county</label>
    <div class="outer-county-select">
      <select id="county-select" data-as="countySelect"></select>
    </div>
  </div>

  <h4 data-as="countyName"></h4>
  
  <results-table data-as="resultsTable">
    Select a county to see detailed results
  </results-table>

</div>

<div class="uncontested-message">
  County-level results are not shown for uncontested races.
</div>

county-detail.js

If you've never thought about how much work goes into displaying election results, this may be a rude awakening for you. Feel free to let your eyes glaze over between the *** PROCESSING START *** and END comments, which mark where we transform the raw AP data into a usable form.

var ElementBase = require("../elementBase");
var Retriever = require("../retriever");

var $ = require("../../lib/qsa"); // querySelector alias
var track = require("../../lib/tracking"); // Google Analytics events

// related components and styles
require("../results-table");
require("../county-map");
require("./county-detail.less");

// load headshots for presidential candidates from a Google sheet
var mugs = require("mugs.sheet.json");

var {
  formatAPDate,
  formatTime,
  groupBy,
  mapToElements,
  toggleAttribute
} = require("../utils");

// generate a color lookup for well-known candidates
var colorSet = new Set();
for (var m in mugs) {
  if (mugs[m].color) colorSet.add(mugs[m].color);
}
var colorKey = Array.from(colorSet);

class CountyDetail extends ElementBase {
  constructor() {
    super();
    // a Retriever is a utility for observing files on S3 for updates
    this.fetch = new Retriever(this.load);
    this.palette = {};
    // listen for events dispatched from the inner map
    this.addEventListener("map-click", function(e) {
      var fips = e.detail.fips;
      track("click-county", fips);
      var elements = this.illuminate();
      elements.countySelect.value = fips;
      this.updateTable(fips);
    });
  }

  static get boundMethods() {
    return ["load", "onSelectCounty"];
  }

  static get observedAttributes() {
    return ["src", "party", "live", "map"];
  }

  static get mirroredProps() {
    return ["src"];
  }

  attributeChangedCallback(attr, was, value) {
    switch (attr) {
      case "src":
        this.fetch.watch(value, this.getAttribute("live") || 60);
        break;

      case "party":
        this.render();
        break;

      case "live":
        if (typeof value == "string") {
          this.fetch.start(value);
        } else {
          this.fetch.stop();
        }
        break;

      case "map":
        var elements = this.illuminate();
        elements.map.src = value;
        break;
    }
  }

  load(data) {
    this.cache = data;
    this.render();
  }

  render() {
    var colors = colorKey.slice();

    var elements = this.illuminate();
    this.classList.remove("uncontested");
    var data = this.cache;
    if (!data) return;
    var { races, test } = data;

    /* *** PROCESSING START *** */

    var party = this.getAttribute("party");
    if (!party || party == "undefined") party = false;

    // filter races by type - no weird alignments
    var [race] = races.filter(
      r => (party ? r.party == party : true) && r.type && !r.type.match(/alignment/i)
    );

    var results = race.results;
    var counties = {};
    var fips = {};
    var totals = {}; // state-wide totals

    var candidates = new Set();

    // aggregate data based on county results
    results.forEach(function(r) {
      var top = null;
      r.candidates.forEach(function(candidate) {
      // create an empty totals object when first seeing a candidate
        if (!totals[candidate.id])
          totals[candidate.id] = {
            first: candidate.first,
            last: candidate.last,
            id: candidate.id,
            votes: 0
          };
        totals[candidate.id].votes += candidate.votes;
        if (!top || candidate.percentage > top.percentage) {
          top = candidate;
        }
        delete candidate.winner;
        candidates.add(candidate.id);
      });
      // counties are only marked as winners when all votes are in
      if (r.reportingPercentage == 100) {
        top.winner = true;
      }
      // add each county to our lookup objects
      counties[r.county] = r.fips;
      fips[r.fips] = r.county;
    });

    if (candidates.size == 1) {
      // suppress display for uncontested races
      return this.classList.add("uncontested");
    }

    // for future map use: determine the palette by statewide total position
    var statewide = Object.values(totals);
    statewide.sort((a, b) => b.votes - a.votes);

    // generate a consistent palette between the table and map
    var palette = {};
    statewide.slice(0, colors.length).forEach(function(s, i) {
      var color = mugs[s.last] && mugs[s.last].color;

      if (color) {
        var index = colors.indexOf(color);
        colors.splice(index,1);

        palette[s.id] = {
          id: s.id,
          first: s.first,
          last: s.last,
          color: color,
          order: i
        };
      } else {
        palette[s.id] = {
          id: s.id,
          first: s.first,
          last: s.last,
          order: i
        };
      }
    });
    
    var index = 0;
    for (var p in palette) {
      var candidate = palette[p];
      if (!candidate.color) {
        candidate.color = colors[index];
        index += 1;
      }
    }

    if (statewide.length > colors.length) {
      palette.other = {
        id: "other",
        last: "Other",
        color: "#bbb",
        order: 999
      };
    }

    /* *** PROCESSING END *** */

    // paint the map with county results
    elements.map.render(palette, race.results, this.dataset.state);

    // generate the county select drop-down items
    counties = Object.keys(counties)
      .sort()
      .map(county => ({ county, id: counties[county] }));
    mapToElements(elements.countySelect, counties, function(data) {
      var option = document.createElement("option");
      option.innerHTML = data.county.replace(/\s[a-z]/g, match =>
        match.toUpperCase()
      );
      option.value = data.id;
      return option;
    });

    // by default, show the largest county in the state
    var maxPop = 0;
    var maxFips;
    results.forEach(function(r) {
      if (r.population > maxPop) {
        maxPop = r.population;
        maxFips = r.fips;
      }
    });

    elements.countySelect.value = maxFips;
    elements.map.highlightCounty(maxFips);
    this.updateTable(maxFips);
  }

  updateTable(fips) {
    if (fips == 0) return;
    var elements = this.illuminate();
    var { resultsTable } = elements;
    var data = this.cache;
    var party = this.getAttribute("party");
    if (!party || party == "undefined") party = false;
    if (!data) return;

    var { races } = data;
    // filter races by type - no weird alignments
    var [race] = races.filter(
      r => (party ? r.party == party : true) && r.type && !r.type.match(/alignment/i)
    );
    var [result] = race.results.filter(r => r.fips == fips);

    resultsTable.setAttribute(
      "headline", 
      result.county + ` (Pop. ${result.population.toLocaleString()})`
    );
    resultsTable.setAttribute("max", 99);
    if (result) resultsTable.render(result);
  }

  static get template() {
    return require("./_template.html");
  }

  onSelectCounty() {
    var elements = this.illuminate();
    var fips = elements.countySelect.value;
    track("select-county", fips);
    this.updateTable(fips);
    elements.map.highlightCounty(fips);
  }

  illuminate() {
    var elements = super.illuminate();
    elements.countySelect.addEventListener("change", this.onSelectCounty);
    return elements;
  }
}

CountyDetail.define("county-detail");

Notes

Retrievers and static data

This component relies on data from two sources. Our interactive template, which forms the basis of the election displays, pulls configuration from Google Sheets. In the code, we load a set of candidate headshots with var mugs = require("mugs.sheet.json");. In other components, static data from Sheets is added to the bundle for things like string constants, but we don't need to do that here.

This component also uses a Retriever class, which is stored on this.fetch in the constructor, to observe the results files that are published to our S3 bucket by the data pipeline. A Retriever targets a single file, which it checks for updates (using the ETag header) at regular intervals — usually every 15 seconds. If it sees a new version of the file, it runs the callback function passed to its constructor, in line with the pattern set by other Observer objects.

In this class, that callback is the load() method, which caches the data and then calls render() to actually set it up. If the element hasn't already injected its internal template, a call to illuminate() does that, and then the component performs a lot of data processing. Finally, it passes that data to the map for painting, and updates the table to match.

The <county-map> interface

You can see the source for <county-map> here, but at a high level, it does three things: it loads a pre-rendered SVG of the state when the "src" attribute is set, it colors in parts of that SVG based on data passed to its render() method, and it shows a popup with details about each county as you mouse over the map.

For this project, we largely standardized on render() as a general-purpose code for "update the page from data" for our display components. It never reached the point that the data-oriented parent components were entirely agnostic about their children, but a consistent method signature made it a lot easier to move between the many components on these results pages.

The map's "src" attribute is passed through the "map" attribute of <county-details>, which itself is set from the page template itself. We made heavy use of attributes as configuration for this project, based on data from the primary calendar, in part because it was easier to verify that future events were configured correctly when we could just view the page source.

The <results-table> interface

Like the map, a <results-table> has a render() method that causes it to update its contents. It also supports an "href" element to show a "more results" link when it's used as an embed, and a "headline" attribute to set its title text. If you look through the source, you'll also see that it uses an EJS template for the actual table contents — many components in the primaries rig perform simple iteration with a matchElements() function that's similar to Radio's matchData(), but for anything more complex we used a real template language.

<results-table> was actually one of two tables that are used whenever you see candidate name and vote totals in the primary election pages. The other was <president-results>, which is basically just the same table but with candidate portraits and the option of wrapping to two columns. From an interface perspective, however, they're basically interchangeable.

Coordinating displays

There are two ways of digging into the county-level results on these pages: you can click on the map, or you can use the drop-down menu to select a county in alphabetical order. Having both navigation mechanisms made sense to us — many people think about finding themselves on a map, but on mobile (or for the spatially challenged) it's easier to use the select box than to poke at the map until you hit the right county.

The select box is the easier of the two to coordinate, in part because it's owned directly by the <county-details> component itself. A listener that calls onSelectCounty() is added in the illuminate() method (which performs lazy templating). We get the value from the select box, which is a county FIPS code, then call updateTable() to re-render the <results-table> and use the map's highlightCounty() method to visually indicate its location.

From the map, the flow is a little more indirect. <county-map> dispatches a "map-click" event when the user touches a county, and highlights the correct location. In our <county-details> constructor, we can listen for this event on the component itself, where it will bubble up from the map. The event includes the FIPS code, which we can use to call updateTable() and set the select box to the correct display.