Monday, September 14, 2020

Predicting Trump's Win in 2016 and Loss in 2020--Allan Lichtman's Success (and Failure)

In September of 2016, Allan Lichtman was one of the few election forecasters who predicted that Donald Trump would defeat Hillary Clinton.  Now, he is predicting that Trump will be defeated by Joe Biden in November.  He announced his new prediction on August 5th in a video produced by the New York Times:

Although it has often been reported that Lichtman was the only expert forecaster who predicted Trump's win in 2016, there were at least three others who made the same prediction, although based on different forecasting models: Helmut Norpoth (political scientist at Stony Brook University), Ray Fair (economist at Yale University), and Alan Abramowitz (political scientist at Emory University).  Moreover, the documentary film-maker Michael Moore also predicted Trump's victory.

Now, Abramowitz agrees with Lichtman in predicting a Biden win.  By contrast, Norpoth and Moore are predicting that Trump will win again this year.  Fair says he cannot make a prediction because his forecasting model "has nothing to say about the effects of pandemics."

This makes me wonder whether a study of the evolutionary history of American presidential elections can support a forecasting model that accurately predicts the election of the next president.  And if this is so, what is the best model?

Lichtman has persuaded me that he has the best model, although I think it needs one modification.  Once that modification is made, it is possible to predict the outcome of almost every presidential election since 1860, including the unusual elections of 2000, 2016, and now this year's election.  This model for forecasting presidential elections based on American political history would be an important part of any biopolitical science of American politics.


Lichtman is a historian at American University.  In 1980, he decided to study the history of every presidential election from 1860 to 1980, and to look for recurrent historical patterns that would predict whether the incumbent presidential party--the party that has controlled the White House--will win the presidential election.  His idea was that voters in a presidential election are mostly judging the past performance of the incumbent party, and that what happens in the presidential campaign--campaign tactics, fluctuating polling, presidential debates, campaign fundraising, campaign advertising, and so on--don't really matter, because all that matters for the voters--the past performance of the incumbent party--has been determined long before the campaign began.  Of course, this contradicts what most political commentators assume, which is that the outcome depends upon the day-to-day events of the presidential campaign.

From his study of the history, as laid out in his book Predicting the Next President: The Keys to the White House (Rowman and Littlefield, 2020 edition), Lichtman decided that there were 13 conditions that favor reelection of the incumbent party.  These 13 conditions could be framed as 13 statements, so that when six or more of these statements are false, the incumbent party loses.

KEY 1  Incumbent party mandate:  After the midterm elections, the incumbent party holds more seats in the U.S. House of Representatives than it did after the previous midterm elections.

KEY 2  Nomination contest:  There is no serious contest for the incumbent-party nomination.

KEY 3  Incumbency:  The incumbent-party candidate is the sitting president.

KEY 4  Third party:  There is no significant third-party or independent campaign.

KEY 5  Short-term economy:  The economy is not in recession during the election campaign.

KEY 6  Long-term economy:  Real annual per capita economic growth during the term equals or exceeds mean growth during the two previous terms.

KEY 7  Policy change:  The incumbent administration effects major changes in national policy.

KEY 8  Social unrest:  There is no sustained social unrest during the term.

KEY 9  Scandal:  The incumbent administration is untainted by major scandal.

KEY 10  Foreign or military failure:  The incumbent administration suffers no major failure in foreign or military affairs.

KEY 11  Foreign or military success:  The incumbent administration achieves a major success in foreign or military affairs.

KEY 12  Incumbent charisma:  The incumbent-party candidate is charismatic or a national hero.

KEY 13  Challenger charisma:  The challenging-party candidate is not charismatic or a national hero.

These thirteen keys fall into three categories.  The first four are the political keys, because they have to do with the political conditions of the election.  The next seven keys (5-11) are the performance keys, because they have to do with the success or failure of the incumbent part in governing.  The last two keys are the personality keys, because they are about whether one or both of the candidates have the attractive personality of a charismatic person or national hero.  Notice that of the twelve keys, only two are about the personal characteristics of the candidates: it's the incumbent party that is being judged, not the candidates.

Although Lichtman does not assign weights to the 13 keys, he does recognize that some of them have greater predictive power than others.  The best single predictor is the contest key 2--whether the incumbent party agrees early and clearly on its nominee.  This contest key calls thirty-four of the thirty-nine previous elections--a prediction rate of 87 percent.  In only four of the twenty-seven elections in which the incumbent party had an uncontested nominee did that incumbent candidate lose--Herbert Hoover in 1932, Richard Nixon in 1960, George H. W. Bush in 1992, and John McCain in 2008 (pp. 25-26).

The second strongest predictor is the short-term economy key 5.  In 80 percent of the cases where there was no recession during the election year, the incumbent-party candidate won.  Even more remarkable is the loss rate: in the nine elections in which there was a recession during the fall campaign, the incumbent party has lost every time (pp. 10, 16, 32-33).  

Although Lichtman generally plays down the importance of public opinion polls, he does admit that such polls can be important for judging the short-term economy key.  What counts is the public perception that the economy is or is not in a recession, and polling might be necessary to determine that.  This could be crucial this year.  The American economy clearly fell into a massive recession in the spring as a result of the COVID shutdown.  But beginning in the late summer, there has been some evidence of recovery.  Trump would like to convince the voters that a dramatic economy recovery has begun only a few months after the downturn.


Lichtman applies his keys model to every presidential election from 1860 to 2020.  In all of the elections up to 1980, he shows how his 13 keys can retrospectively predict the outcome of each election, because he can explain that the incumbent-party candidate lost whenever 6 or more of the keys were forfeited.  For the elections from 1984 to 2016, he can show that he was correct in prospectively predicting the outcome months before the election by applying his keys approach.  Now he hopes to show that he has correctly predicted Biden's victory in November.

Consider, for example, how Lichtman explains the reelection of Abraham Lincoln in the presidential election of 1864.  Lincoln was running as the sitting president of the incumbent Republican Party, and he had not faced any challenger for the nomination, so the incumbent party had keys 2 and 3.  Although the Republicans had lost a few seats in the House in the 1862 mid-term elections, they still had a 55 percent majority, which was greater than their 48 percent of seats prior to 1860, so they had key 1 (incumbent-party mandate).

The Democrats nominated George B. McClellan, the former general-in-chief of the Union armies whom Lincoln had fired in 1862.  McClellan was not a charismatic personality, so the Republicans had key 13 (challenger charisma).  By contrast, most of us would assume today that the Republicans had the advantage of Lincoln's incumbent charisma (key 12).  But Lichtman rightly notes that Lincoln was not known during his lifetime as a charismatic politician, but as a remarkably awkward-looking person in public.  He became charismatic only after his assassination, when he became the epic martyr of American history.  So in 1864, the Republicans lost key 12.

The Republicans also lost key 6 (long-term economy), because the economic performance during the Civil War was slower than it had been in the two previous presidential terms (1852-1860).  But since the economy was improving in the election year of 1864, the Republicans had key 5 (short-term economy).

Obviously, in the midst of the Civil War, there was great social unrest, so the Republicans lost key 8.  So without three keys (6, 8, and 12), the Republicans could not afford to lose three more keys without losing the presidential election.

Lincoln and the Republicans had been forced by the war to make some major changes in national policies--such as a national banking system, a nation-wide draft, and the Emancipation Proclamation--so the Republicans had key 7 (policy change).  They also had key 9 (scandal), because the Lincoln Administration had avoided the taint of major scandals.

As one would expect in the only presidential election held during a civil war, the crucial keys for Lincoln and the Republicans were military.  Could the Union armies achieve military success (key 11)?  Or at least avoid any major military failure (key 10)?

In the spring and summer of 1864, the war appeared to have reached a stalemate; and consequently Lincoln's prospect for reelection appeared dim.  Some Republicans were so dissatisfied with Lincoln's military leadership that they were recommending John C. Fremont--who had been the first Republican candidate for the presidency in 1856--to run again.  If Fremont had run as a strong third-party candidate, and if there had not been any major Union victories on the battlefield, the Republicans would have lost three more keys (4, 10, and 11).  With the loss of six keys, Lichtman's model would have predicted Lincoln's defeat.

But then, on September 3, the news arrived in Washington that General Sherman's army had taken Atlanta.  And by October 19, General Sheridan had driven the rebels from Virginia's Shenandoah Valley.  Now a Union victory in the war appeared likely.  Fremont lost interest in running as a third party candidate.

Having lost only three keys, Lincoln defeated McClellan 55 percent to 45 percent.


In a similar way, Lichtman can use his keys system to explain the outcomes in all of the presidential elections from 1860 to 1980.  Even more impressive than this historical retrospective, however, is that he seems to have predicted each presidential election from 1984 to the present.  And sometimes he has predicted victory for a candidate who started the campaign far behind in the polls.  

For example, in 1988, George H. W. Bush (the former Vice President under Reagan) was running against Michael Dukakis (the Governor of Massachusetts); and Bush in the spring and summer was far behind Dukakis in the polls.  By the end of July, Dukakis was ahead by 17 percent in the Gallup Poll.  But Lichtman saw that the incumbent Republican Party had lost only three keys.  Their candidate was not the sitting president, and so they lost key 3.  They had also lost key 7, because there had not been any major policy changes in Reagan's second term.  And they lost key 12, because Bush did not have Reagan's charisma.  With only three keys turned against Bush, Lichtman could predict his victory.  And indeed Bush carried 40 states and 53.4 percent of the popular vote.

But then a greater challenge to Lichtman's forecasting model came in 2000.  The incumbent-party candidate Democrat Al Gore (the former Vice President under Bill Clinton) was running against Republican George W. Bush (the Governor of Texas).  Lichtman saw the Democrats as losing five keys--3 (incumbency), 7 (policy change), 9 (scandal), 11 (foreign or military success), and 12 (incumbent charisma).  Since this fell one key short of the six keys threshold for defeat, he predicted that Gore would win, although perhaps with a small vote margin.

While Gore did win the popular vote--by about 544,000 votes--Bush won election by winning in the Electoral College with a narrow margin (271 votes to 266 votes for Gore).  The electoral college vote was decided when Florida's 25 electoral votes went to Bush.  The U.S. Supreme Court halted the recount of votes in Florida with Bush leading Gore by 537 votes out of almost 6 million votes counted.

Although it seemed that Lichtman had failed in his prediction, he has insisted that his prediction was correct, because he correctly predicted that Gore would be the popular vote winner!  He had always assumed that his prediction of the popular vote winner would almost always coincide with the Electoral College outcome, because since 1860 there had been only one case when the winner of the popular vote lost the Electoral College tally--when Benjamin Harrison defeated Grover Cleveland in 1888.

Amazingly, this happened again in 2016, except that this time, Lichtman predicted Trump to be the winner, and Trump won the Electoral College while losing the popular vote to Clinton by almost three million votes.

This discrepancy between the popular vote and the Electoral College has created a crisis of legitimacy for the American presidency because it creates doubt that the presidential election shows the consent of the people.  Lichtman thinks his keys model can handle this problem without any fundamental change in the 13 keys.  But I think that he needs to add a new key 14.

This problem arises from the fact that many of the likely voters for the Democratic Party are highly concentrated in large urban areas in populous states in the northeast and west--such as New York and California--while the likely voters for the Republican Party are scattered widely over rural and small town areas in states in the Midwest, South, and Mountain West.  Consequently, the Democratic candidate can win  all the electoral votes of the states of New York and California with an extra margin of 5 to 6 million votes, which in effect become "wasted votes."  But then the Republican Party has far fewer wasted votes.  In 2016, not a single state gave Trump a margin of over one million votes.  His margin in Michigan was only about 10,000 votes.  So in any close election, the Democratic candidate can win the popular vote while losing the Electoral College count.

So what we need, I suggest, is one more key: 

KEY 14 The Electoral College:  The likely voters for the incumbent party are evenly distributed across the states so as to minimize wasted votes in the Electoral College system. 

Adding that key to Lichtman's model would have allowed him to predict the Electoral College victories of Bush in 2000 and Trump in 2016, despite Gore and Clinton winning the popular vote.

In 2000, Lichtman saw that Gore, as the incumbent-party candidate, had 5 keys against him, one short of the threshold for defeat, and so Gore's win seemed predictable.  If Lichtman had applied my key 14, he would have seen that the Democrats were likely to have a lot of wasted votes in states like New York and California, and in a close election, this could cause them to lose in the Electoral College.  Losing key 14, Gore would have been at the 6 keys threshold for a loss.  Therefore, Lichtman could have predicted Bush's win.


Having my key 14 would also have helped Lichtman in his prediction for 2016.  In his interview with the Washington Post published on September 23, 2016, he predicted that Trump would win, because Clinton, as the incumbent-party candidate, had lost 6 of the keys to the White House--1, 3, 4, 7, 11, 12.  One of those keys was in question, however--the third party key 4.  Gary Johnson, the Libertarian Party candidate, was polling as high as 12 to 14 percent.  If Johnson's final vote tally were to be as high as 5 percent, which seemed likely to Lichtman in September, that would pass the threshold for Clinton losing key 4.  And yet Johnson actually received only about 3.2 percent of the vote; and so Lichtman should not have counted the third party key as lost to Clinton.  In that case, with only 5 keys against her, he should have predicted Clinton to be the winner.  But if he had had key 14, he could have seen Clinton as losing 6 keys; and he could have predicted her loss of the Electoral College count, even while winning the popular vote in a close election.

Michael Moore came closer than Lichtman in predicting the exact outcome of the 2016 election.  Moore foresaw that Trump could win in the Electoral College by carrying Michigan, Ohio, Pennsylvania, and Wisconsin along with the states won by Mitt Romney in 2012.

In Predicting the Next President, Lichtman says that his 13 keys "focus on national concerns such as economic performance, policy initiatives, social unrest, presidential scandal, and successes and failures in foreign affairs," and thus "they predict only the national popular vote and not the vote within individual states," and therefore there was no way that he could have predicted Gore as the popular vote winner losing in the Electoral College in 2000 (p. xi).  But if that is true, then his prediction in 2016 was mistaken, because he did not predict Clinton's winning of the popular vote.

In an interview on November 18, 2016, eight days after the election, Lichtman said that while he was not going to change his model, he had changed his interpretation of what it predicted; so that now he would predict the winner of the election in the Electoral College regardless of whether this was the popular vote winner.

In his 2020 edition of Predicting the Next President, Lichtman writes: "In 2016, I made the first modification of the keys system since its inception in 1981.  I did not change any of the Keys themselves or the decision-rule that any six or more negative keys predict the defeat of the party holding the White House.  Rather, in my final forecast for 2016, I predicted the winner of the presidency, e.g. the Electoral College, rather than the winner of the popular vote.  In an interview with the Washington Post on September 23, 2016, I predicted that Donald Trump would win the presidency in November, after a sixth key turned against the incumbent Democrats" (191).

This passage is strange for three reasons. First, it contradicts the passage quoted above that his keys "predict only the national popular vote."  

Second, in his 2016 interview with the Washington Post, he spoke of a "Trump victory," but he did not say that this would be a victory in the Electoral College that might allow Clinton to be the popular vote winner.

Third, in saying that his "modification of the keys system" does not change any of the keys themselves or the decision rule, he leaves the reader confused as to how his "modification" allows him to predict a winner in the Electoral College who loses the popular vote.

Wouldn't it be better for Lichtman to either admit that his keys predict only the popular vote winner and not the Electoral College winner, or to adopt my key 14 so that he can anticipate a discrepancy between the popular vote and the Electoral College?


Consider now how my proposed system of 14 keys would apply to the 2020 election.  First, we could judge that Trump as the incumbent-party candidate wins key 14, because his likely voters are distributed evenly enough across the states to minimize any wasted votes in the Electoral College system.

We might then be persuaded by Lichtman's judgment that Trump's party has lost 7 of the keys, and so he is likely to lose the election to Biden.

Trump has lost the incumbent-party mandate key 1, because after the 2018 mid-term elections, the incumbent Republican Party did not hold more seats in the House than it did after the 2014 mid-term elections.  I have written about the importance of this here.

Trump has lost both of the economic keys (5 and 6), because of the economic decline both short-term and long-term that has come from the COVID economic shutdown that Trump initiated in March.  I have written about that here and here.

He has lost the social unrest key 8, because of the mass street protests and violence that has spread around the country since May.  I have written about this in my posts on nonviolent and violent resistance to perceived injustice here and here.

He has lost the foreign or military success key 11, because he has not achieved any major success in foreign or military affairs.

Since there have been many major scandals connected to Trump and his administration, he has lost key 9.

Finally, he has lost key 12, because although his grandiose showmanship appeals to his most fervent supporters, he lacks the charismatic personality that would appeal broadly to the whole electorate.  This shows the fundamental weakness in Trump's governance that comes from his bad character--his lack of moral or intellectual virtues.  I have written about that here, and here.

If this is true, then Biden is likely to beat Trump.  And this will have nothing to do with Biden's campaign, because all that matters is the voters' judgment of the performance of the incumbent Republican party over the past four years.


We are left with some big questions about the American system of presidential election, which is suggested by the Electoral College key 14:  Why should there ever be a discrepancy between the popular vote winner and the Electoral College winner?  Is the Electoral College winner a better expression of popular consent than the popular vote by itself?

We could change this situation by abolishing the Electoral College, as has often been proposed.  Or we could adopt the procedure for allocating Electoral College votes that is followed in Nebraska and Maine: two of the electoral votes go to the statewide popular vote winner and the others to the winners in each of the congressional districts.  So, for example, Nebraska has 5 electoral votes (corresponding to Nebraska's two senators and three congressman).  In 2008, John McCain won 4 of those electoral votes, because he won the popular vote statewide and in two of the congressional districts.  Obama won one electoral vote, because he won in one of the congressional districts.  If this procedure were followed in all the states, there would be almost no chance of any divergence between the popular vote winner and the winner in the Electoral College.

No comments: