During this year's big Iowa caucus back in February, after each meeting had finished dividing up the room, barn or gymnasium and the organizers had counted the supporters for each candidate, the precinct captain in each did something different: they pulled out their phones.
Rather than begin tabulating the results through a complex formula, they used a specially-designed smartphone app to send the results back to party headquarters.
The Iowa caucuses have always been a major occasion in American democracy. For both the Republican and Democratic parties, the Hawkeye state uses its pole position in the electoral calendar to allow voters to cast the first judgements upon what usually is – and this year certainly was – a wide array of presidential candidates.
Caucuses are not simply an act of drawing an "X" on a ballot paper or pulling a lever on a machine. They are politics done the old fashioned way – as it has been for over 100 years: a group of people getting together in a room to debate and express their preferences in public.
Iowa has been first since 1968, and it is easy to assume that Iowans were surely doing the same thing that their parents, and their grandparents, had done before them. Well, not quite.
This year's caucuses came with a technological twist, a stark reminder of how new tech has turned a war for your vote into a war for your data.
21st Century caucusing
Back to that smartphone app. The app – or rather, apps – were built at Microsoft by a team led by Stan Freck, Senior Director for Campaigns and Elections. The company worked with both main parties to develop apps for use in Iowa – across iOS, Android and Windows Mobile – as well as back-end services to collect the data and even distribute it to news organizations.
This is just a glimpse of how elections of the future could be made more efficient.
Rather than spend hours waiting for each precinct to phone in the data and tabulate it manually, Microsoft's system accepts the data from the apps and does the calculations in real time. And then, once it had been checked by party headquarters, the data could be beamed to the web and the major news organizations using an application programming interface, or API.
That way, CNN, for example, could take the data and throw it up on a screen within seconds.
We work with Coke and we work with Pepsi. Our approach will be to not play, not pick favorites. – Microsoft's Stan Freck
I asked Freck whether there were any redundancies in place: what if the whole system had melted down? Would it have been days until we knew of Ted Cruz's surprise victory? According to Freck, Microsoft supplements the system with a phone-based, interactive voice response system ("Press 1 to…" and so on), which some precincts opted to use instead of the app.
But, even this system still fed into Microsoft's central counting system, ensuring that votes could be counted quickly. And, in a worst case scenario? The parties both still required precinct captains to fill in paper forms the real old-fashioned way – the ultimate insurance policy.
But there wasn't a problem – and that perhaps shouldn't have been surprising given that, as Freck says, "Actually taking somebody's vote electronically isn't that complicated – the amount of data you're talking about for however many couple of hundred million Americans is still not the scale of data that our company and other companies with cloud services deal with when you're doing financial transactions for example. They're just nowhere close."
Back at base, Microsoft also applied some sanity checks before sending the results out to the news networks. For example, comparing the number of people who the captains had said turned up versus the number of actual votes; if there is a big discrepancy, it suggests that data has been entered incorrectly.
What all of this meant was that, by 11:30pm ET, Ted Cruz was able to confidently stand at a podium and give a victory speech far earlier than would have been possible in previous years.
Sadly, the Democrats were left waiting until the early hours of the morning – due to a painfully close contest (one that in six precincts was allegedly decided by coin-flip).
Even with modern technology, it is easy to understand why the networks were cautious before calling it.
Overall, it appears that Microsoft's Iowa experiment was a success. It was a bold use of new technology – but one that, in 2016 at least, will not be repeated. Because each state has its own byzantine electoral rules, re-using the app for other caucus states, like Nevada, or primary states, like New Hampshire, is pretty much impossible.
But, now that Microsoft has a proof of concept, will we see the same tech deployed more widely in 2020? Perhaps, though Freck diplomatically says, "We aren't at a point, as a company, where we have stated ambitions around that yet."
Staying nonpartisan
Microsoft's involvement in the Iowa caucus was strictly non-partisan, working with both parties on what Freck calls the "infrastructure" of running an election rather than on anything which might count as campaigning. (He points out that Iowa is unique, in that both the red and blue teams work together more closely than perhaps any other state in coordinating what are, technically, two completely separate caucuses.)
The reason behind this is simple: money. Microsoft wants to sell to both sides – and that requires trust – something which Freck points to other Microsoft clients for examples of.
"We work with Coke and we work with Pepsi as customers", he explains. "We work with Ukraine, and we work with Russia."
He believes that Microsoft's vast scale actually helps reassure politicians in this case that the company will play fairly. This contrasts with smaller firms, which – due to their smaller size – might necessarily put staff who are working with two rival clients closer together.
"Our approach will be to not play, not pick favorites, not to play that game per-se," Freck says. "There's certainly this ability to set up 'firewalls' between teams, so that one entity that you're trying to assist doesn't have a fear that their secret sauce, whatever that may be, isn't going to move across your organization and end up in the other people's hands".
But, will this always be the case? Amusingly, the latest season of Netflix's House of Cards features a storyline involving President Frank Underwood using a search engine to data-mine information about how best to target people in an election – to win him an advantage in his bid for re-election.
Could this really happen? Could Microsoft, or perhaps Google or Facebook, ever team up to help a specific candidate win?
"It's not that it's not [technically] feasible [but] it's a great game to play for four years, maybe eight in the US... and then suddenly it comes back to haunt you", Freck jokes.
But this isn't to say that all companies remain non-partisan: some tech companies – which I'll touch on in a moment – have been explicitly created to support the election of the candidates that hire them.
The data campaign
Data has played a role in election campaigns going back decades, but it is only since the turn of the millennium that technology has grown truly sophisticated. The ill-fated Howard Dean campaign of 2004 is often cited as the turning point, when digital started to be taken seriously, and Obama's 2008 election was testament to the power of micro and predictive targeting.
What this means is that, in 2016, if a campaigner knocks on your door, the chances are they already know who you are, details about your family and who you supported in previous campaigns. They probably also know who you are thinking of voting for this time, too, and will tailor their "sales pitch" to appeal directly to you.
A high quality processed voter file will always be an element of any political campaign's data strategy. – campaign data firm L2's Paul Wescott
The reason they know this is because of a number of interlinked technologies. The first problem solved with tech had long been a frustrating one: if candidates want to know who is on the electoral roll of each state, each state party has its own records, with its own notes.
And, worse still, in a presidential campaign after two years of collecting so much intelligence on voters, the day after the election, the campaign is wound up and the data is made redundant. That is, until the next election, when all of the new campaigns start from nothing again.
On the Democratic side, as chronicled in The Victory Lab by Sasha Issenberg, campaigners Laura Quinn and Harold Ickes came up with an ingenious solution, which later became known as Catalist. Founded in 2006, Catalist is a "data consortium" set up as a non-profit by leaders of a number of US progressive organizations. The idea is that, as a "data warehouse," it can provide the data for a number of partisan actors, which won't die with any individual campaign.
Essentially, Catalist has a massive database of voters from all across the country – and the way it has built it up is genius. At the start of a campaign, a candidate will lease all of the data from the company on the proviso that, after the election, they will feed back into the database the new data that they have collected.
This means that the 2016 Hillary Clinton campaign does not have access to just the data they collected in 2008 – they also have Obama's voter data from 2008 and 2012, too. Super PACs, unions and other Democrat-affiliated organizations are clients of Catalist, too, enabling the whole Democratic world to pool resources and share intelligence.
This rich data, and the ability for Catalist to target specific individuals pays off in dividends, too, as it helps make spending those precious campaign dollars more efficient. Michael Frias, Catalist's Chief Client Officer, tells me about how it worked for the Super PAC which is backing Hillary:
"Ready for Hillary was interested in improving their fundraising ROI [Return on Investment]," Frias says. "We worked together to develop a custom donor model by starting with Democratic voters who were older and modeled as relatively likely to donate generally. By using the results of that first mailing to inform a custom model, we were able to dramatically increase their response rate and average donation amount, and to lower their cost to acquire a new donor."
This is a fine example of how data can also be predictive. Rather than simply tell campaigns about individuals, data can also be used to make assumptions about you. In the above case, the people targeted weren't just previous donors, but people who matched the profile of the sorts of people who would donate. So, even if the campaigns don't know anything about you specifically, they can make a good guess.
There is virtually no limit to how far this can go as the technology continues evolving. – Moonshadow Mobile's Bob Lee
It isn't just the Democrats who have a system like this. The GOP is also wise to it, too, with a number of data warehouses working for its Republican candidates and causes, such as DataTrust and Themis. The latter of which was founded by the infamous Koch brothers to perform a similar function to Catalist. And there are also non-partisan equivalents, which are happy to lease voter data to the highest bidder.
One example of the latter is L2, which has reportedly worked with both the Trump and Sanders campaigns in this year to supply its data and numerous visualization and targeting tools. Paul Wescott, the company's Director of Marketing & Business Development, kindly supplied me with what he calls the "data dictionary," the list of what sorts of data L2 holds, and it is staggering.
In addition to data you might expect, like names, birthdates and addresses, L2 will tell campaigns how much you paid for your home, whether you have children, your estimated net worth and income, whether you have a cat and even if you like to read science fiction – in addition to dozens of other similar, weirdly specific things.
The reason the data is so rich is, because data warehouses are able to not just use data collected by political campaigners, they can also match you up with any commercially available data. The classic example of this is that, if you subscribe to a gun magazine, then the publisher can sell this fact to a data warehouse, as it is pretty good proxy for reaching a certain type of voter.
Ground game
Having all of this data wrangled is only half the battle: it is making use of it that counts. And this is where modern apps enter the equation. L2, for example, has partnered with Moonshadow Mobile to build a "Ground Game" mobile app (depicted below) which will give activists on the ground a leg up when collecting data and mobilizing campaign supporters.
"So many elections are determined by such thin margins that successfully micro-targeting voters and then getting those targeted voters to the polls is the difference between winning and losing.", Moonshadow President Bob Lee explains via email. "We've seen instances where our Ground Game clients have won closely-contested elections (for example at the State Legislature level) by only a few thousand or even a few hundred votes and they've credited our software/apps with making the difference in efficiency between their winning campaign and their opponent's losing campaign."
Moonshadow's software enables precinct captains to allocate lists of houses that need to be visited by activists, and then the activists can use a mobile app to feed data back to base in real time. As the activists are directed by the app, before they knock on a door they already have a pretty good idea about who is going to answer, so they'll know whether to mention second amendment rights to a gun magazine subscriber, or NASA to a science fiction reader.
The data-fraught future
So, this is what's happening now, in 2016, but what does the future hold for data's role in our elections?
Microsoft's Freck seems to think that it is inevitable that the electoral process will get more hi-tech.
"I'm looking at my cellphone sitting here at my desk," Freck says. "I expect to be able to pick that thing up and do my banking, do my contacts with people, do my communications [and] hopefully in time be able to do most government services."
Though, he also points out that one of the biggest challenges isn't on the technology side, but the legal side given the patchwork of different electoral laws in different states. Doing something radical, like electronic voting, would be hard to pull off knowing that.
On the campaigning side, though, it seems that the data people are just getting started. L2's Paul Wescott thinks that capturing data on the technology voters use will be important to future campaigning.
"A high quality processed voter file will always be an element of any political campaign's data strategy," Wescott says. "The greatest changes will come in what data in addition to phones, address, email, cookies, device IDs, etc needs to be added to make sure the file can be effectively used for outreach."
Wescott adds that improving the accuracy of the voter file data is also important, naturally.
For example, making sure that each voter's voting history, age, gender, party affiliation and even their social networking usernames are right will help campaigns ensure they are targeting the right people.
Moonshadow's Lee, meanwhile, argues that it isn't just about the data – but about the analysis, too.
"The real Holy Grail here, in my opinion, is data scientists continuing to expand and refine the capabilities of their algorithms to 'out-model' the competition regarding identifying precisely the right voters," Lee says. "The better the modeling, the greater chance of success. There is virtually no limit to how far this can go as the technology continues evolving."
And, perhaps most intriguingly, Catalist's Frias thinks that models will become better at predicting the future.
"Currently, most targeting considers the electorate in the current moment," Frias says. "Models make predictions based on voters' current attributes and behaviors, but now that we have been compiling civic engagement for a decade, [data such as] how people change over time, is proving to be even more important for understanding and predicting election and issue engagement."
That point alone perhaps illustrates just how big a role technology plays in elections now. Who knows, when the next presidential cycle begins, the candidates could already not just know who you are and how you voted last time, but how you'll probably vote on the 3rd of November 2020 – even if you don't yet know yourself.
Lead Image Credit: iStock Photo / EdStock
