Forecasting the 2020 Elections and the Predictive Power of Midterms

By Edward Hamburg

When analysts feast on data as they project the outcomes of national elections in the United States, results from presidential-year contests are considered steak while those from midterms are dismissed as chopped liver. With no Presidential candidates heading party tickets, these “off-year” elections ordinarily get two-thirds the voter turnout and a fraction of the media attention of “on-year” contests. Few history books tell detailed and dramatic stories about midterm elections. At best, they are appreciated as measures of the prevailing popularity or approval of sitting presidents, with positive assessments usually determined by how little support their party loses after occupying the White House the previous two years.

But midterm elections are not electoral chopped liver. Research shows how presidential-year contests are comparatively more sensitive to the short-term effects of particular personalities and dramatic events, often overstating the amount of fluctuation in voter behavior over time. Midterm elections, by contrast, are less affected by such short-term factors and better reflect more enduring voter party loyalties. These qualities enable midterms to reveal historical electoral continuities as well as historically meaningful change. They have validated and reinforced the most important decisions ever made in American presidential-year elections. They may also, as this study suggests, be critical harbingers of electoral things to come.

(Note: The initial research in this area was presented by Angus Campbell, “Voters and Elections: Past and Present,” Journal of Politics 26 (1964): 749–752. A summary of other such studies can be found in Edward Hamburg, “Anatomy of a Realignment: The Dynamics of Electoral Change in America, 1874–1906” (Unpublished Ph.D. dissertation: The University of Chicago, 1982): 14–16).

This analysis extends from an earlier work entitled “Why Democrats Win in 2020” (posted at Medium on 18 March 2020), which projects results in the forthcoming November elections for President and the U.S. House of Representatives based on distinctive changes in voting behavior observed between the 2016 and 2018 Congressional contests. It focuses on the very different declines in the number of votes cast for the major party Congressional candidates: over 12 million fewer votes for Republicans in 2018, or an almost 20% decline, compared to just 1.2 million fewer votes for Democrats, a drop of only 2%. This disparity drove the shift nationwide from a Republican vote margin of 1.1% in the 2016 House elections (a gap of 1.4 million votes) to a Democratic vote margin of 8.7% in 2018 (a gap of 9.7 million votes). To further examine the effects of the disparity, a measure was created called the “Democratic Mobilization Margin,” which subtracts the Republican percentage change in votes from the percentage change for Democrats. This calculation is described in Table 1 below.

The study concludes that the Democratic Mobilization Margin of 17.5% in 2018 — the highest such score achieved since the turn of the century and the third highest seen in over nine decades — indicates the very real possibility of a decisive Democratic victory in the November 2020 popular vote as well as a reasonable vote margin in the Electoral College.

But can differential changes in the number of votes cast for Republican and Democratic Congressional candidates between a presidential-year and its subsequent midterm election really reliably predict the outcome of the contests for Congress and the Presidency two years later?

Before addressing this question, it’s useful to understand the logic underlying the response.

This analysis is based on the contention that long-term electoral success in the United States is achieved by political parties that more effectively:

• expand their audience of supportive voters, both by convincing some number of swing voters to stop swinging as well as recruiting citizens who are not yet regular participants in elections; and
• mobilize (or keep from demobilizing) these new supporters, along with their base constituencies, during election periods.

Audience expansion is an essential part of a party’s strategic plan to gain and retain power. If this expansion proves difficult, then the more a party must resort to rule manipulation (voter suppression, gerrymandering congressional districts), obfuscation (especially with facts relevant to critical issues), and outright deception (usually elaborated with racist or nationalist appeals). The ability to differentially mobilize voters is a party’s primary tactical imperative; nothing is more important than getting more of its supporters to the polls than the opposition — or motivating comparatively fewer from not voting.

The Democratic Mobilization Margin is proposed as an empirical predictor of future electoral outcomes because it quantifies the dynamics of both expansion and mobilization. Positive scores on the measure indicate the Democrats out-expanded and out-mobilized the Republicans from one election to another. If neither party was effective at expansion, then positive scores show the Democrats better mobilized their base of regularly supportive voters, or kept fewer of them from demobilizing, than the Republicans. Negative scores, of course, indicate the reverse conditions in favor of the GOP.

By computing the Democratic Mobilization Margin as the changes from presidential-year elections to the midterms held two years later, the impacts of expansion and mobilization are evaluated between high-intensity contests and those in which intensity is tempered, between elections more affected by personalities and events and those more resistant to such short-term factors, and between elections about which much history is written and those unaccustomed to exhibiting dramatic change. An important possible predictor of future votes should consider both the noise and signal endemic to American electoral outcomes over time.

The following analyses determine how well the Democratic Mobilization Margin, starting with the two-year change between the 1924 (presidential-year) and 1926 (its subsequent midterm) elections and extending through the 2012 and 2014 contests, is predictive of:

• Congressional Democratic Margin Change: the four-year difference, from one presidential-year election to the next, in the margin between the Democratic and Republican percentages of Congressional votes, starting with the change between the 1928 and 1932 elections and extending through the 2012 and 2016 contests; and
• Presidential Democratic Margin Change: the same difference except using returns from Presidential elections.

Table 2 below provides examples of the margin change computations in the 2000–2004 and 2004–2008 elections, with the Democratic Margin Change scores bolded in the bottom right box in each display.

Chart 1 below shows the Democratic Mobilization Margin scores for each midterm election from 1926 to 2018. The pattern of scores is a veritable journey through 92 years of American electoral history.

The highest scores for Democrats were recorded in 1930 (with the onset of the Great Depression) and in 1974 (during the Watergate Crisis); smaller but significant spikes appear in 1958 (economic recession), 1982 (economic recession), 2006 (unpopular wars), and, of course, 2018 (Donald Trump). The highest scores for Republicans were recorded in 1938 (Great Depression deepened), 1946 (Truman unpopular and labor unrest),1966 (Vietnam War escalates and race riots), 1994 (the Gingrich upheaval), and 2010 (the Tea Party revolt). Scores in 1926 of -1.7%, -1.0% in 1934, -0.4% in 1990, and -1.8% in 1998 are too low to clearly appear on the chart.

Note that Democrats achieved positive scores in only 9 of the 24 midterm elections (37.5%), even though they held the White House as many years during this timespan as the Republicans. That Democrats achieved positive scores on this measure only 2 times in the 32 years since 1986 speaks to the eventual dissolution of the New Deal coalition into a changing but unstable supportive base that only showed up in 2006 to create the tide Obama first rode to the Presidency — and in 2018.

Chart 2 below shows that the Democratic Mobilization Margin is strongly correlated with scores on the Congressional Democratic Margin Change measure — indicating that differential changes in the number of votes cast for Democratic and Republican Congressional candidates between presidential-year and their subsequent midterm elections can, with some degree of accuracy, predict the outcomes of Congressional elections held two years later. The Pearson correlation coefficient (R) of .81 is strongly statistically significant. Only 3 of 23 predictions were off by 10 or more percentage points, and only 7 by more than 5.

Examination of the statistically significant predicted misses once again involves recalling some of the most important moments in American electoral history. Elections where the actual results were much higher than predicted for the Democrats include:

• 1932, when Democrats took total control of the US House and Franklin Roosevelt entered the White House, launching his dramatic first 100 days that would evolve into the New Deal;
• 1948, when Democrats surprisingly regained control of the House behind Harry Truman’s unexpected, and famously unpredicted, (re)election to the Presidency; and
• 1964, a year after the assassination of John Kennedy, reversing his party’s lackluster performance in the 1962 midterm to gain the super-majority in the House that went on to launch Lyndon Johnson’s Great Society programs.

The contests where the actual results were far lower for the Democrats than predicted include:

• 1972, when Democrats famously self-destructed and were buried in the Republican landslide that reelected Richard Nixon;
• 1980, when Republicans rode the coat-tails of Ronald Reagan in his first run for President; and
• 1984, when Republicans rebounded from losses in the 1982 midterm with Reagan’s reelection by a wide margin.

Chart 3 below shows that the Democratic Mobilization Margin is correlated, although less strongly, with scores on the Presidential Democratic Margin Change measure — indicating that differential changes in the number of votes cast for Democratic and Republican Congressional candidates between presidential-year and their subsequent midterm elections can, but with less accuracy, predict the outcomes of Presidential elections held two years later. The Pearson correlation coefficient (R) of .58 is statistically significant, but the predictions in 10 of the 23 elections were off by more than 10 percentage points, and 16 by more than 5.

That more error is involved with predictions of future Presidential elections than Congressional contests is no surprise; the broader spread of points on this chart compared to the previous display is just another example of the greater variability historically shown in Presidential voting. Charts 4 and 5 below also provide some statistical support: both the mean margin change and standard deviation in Congressional elections are significantly lower than in Presidential voting.

(Note: Consistent with previous research, the mean Congressional Democratic Margin Change from one midterm election to another is even lower: 0.459%, with a standard deviation of 9.5%).

Yet despite these differences in variability, the Congressional and Presidential Margin Change measures are highly correlated (R=.84), indicating that while the size of the margin changes in Congressional and Presidential elections may differ, they rarely move in opposite directions. Democrats infrequently improve their margin in Congressional voting but not in the Presidential vote, and vice-versa. As shown in Chart 6 below, such opposite-directional elections occurred only 5 of 23 times over 92 years, when:

• Truman in 1948 won with a lower margin than Roosevelt did in 1944, but the percentage of Democratic Congressional votes improved;
• Eisenhower in 1956 won reelection by a larger margin than in 1952, but the percentage of Democratic Congressional votes improved;
• Reagan was reelected in 1984 by a larger margin than in 1980, but the percentage of Democratic congressional votes improved;
• Clinton won in 1992 (remember the impact of Ross Perot), but the percentage of Democratic Congressional votes dropped; and
• Clinton was reelected in 1996 by a larger margin than he had in 1992, but the percentage of Democratic Congressional votes dropped again.

Thus even with this diminished accuracy, midterm Democratic Mobilization Margins are shown to be useful predictors of Presidential as well as Congressional voting results in elections two years in the future — understanding that estimates of the former will have much higher associated error margins than the latter.

For purposes of illustration, and returning to the original focus of this analysis on the forthcoming 2020 elections, presented below are the point estimates (specific predictions) and associated standard error (the probable amount these estimates could be off the mark) for the Congressional and Presidential contests in November.

For the Congressional vote, the constant and slope from Chart 2 as well as the 2018 Democratic Mobilization Margin are placed in the following formula:

9.6% = 1.336 + (0.474 * 17.5%)

where 9.6% is the predicted Democratic Margin Change in 2020. The standard error of this prediction is 6.2%, putting the low end of the predicted range for 68% of cases at 3.4%, and the high at 15.8%.

For the Presidential vote, the constant and slope from Chart 3 as well as the 2018 Democratic Mobilization Margin are placed in the following formula:

11.4% = 2.171 + (0.527 * 17.5%)

where 11.4% is the predicted Democratic Margin Change in 2020. The standard error of this prediction is 13.4%, more than two times its Congressional counterpart, putting the low end of the predicted range for 68% of cases at -2.0%, and the high 24.8%.

These calculations predict that:

• the percentage of Congressional votes nationwide cast for Democratic candidates in 2020 will increase 9.6 percentage points from their -1.1% margin in 2016 to 8.5%, with a likely range from a low of 2.3% to a high of 14.7%; and
• the percentage of Presidential votes nationwide cast for the Democratic candidate in 2020 will increase 11.4 percentage points from their 2.2% margin (yes, Clinton won the popular vote) in 2016 to 13.6%, with a likely range from a low of 0.2% to a high of 27.0%.

Under none of these estimated circumstances do Democrats lose control of the House or fail to capture a majority of the two-party vote for President. Both contests would be nail-biters in the few (16%) of possible outcomes beyond the low-ends of their ranges, and both would approach or surpass historic performances in the equally unlikely few possible outcomes beyond the range high-ends.

But anyone who makes predictions should think about the factors that could completely throw off even their most carefully developed estimates of future outcomes. For example, the presented November 2020 predictions could be materially affected by:

• The coronavirus. Such dramatic events can have very different electoral effects. There are any number of possibilities, including:
• The crisis is cast as a war requiring a wartime footing, with Trump cast as the heroic leader of a unified national effort (as with H.W. Bush during the Gulf War and George W. Bush after 9/11), enabling Republicans to ride this new wave of sentiment to reverse or overwhelm the Democratic mobilization developed over the last three years; or
• The crisis is cast as a disastrous event to which national leaders responded ineffectively, with Trump cast as an inept chief executive of a feckless administration, further reinforcing if not accelerating the already well developed Democratic mobilization; or
• Voter positioning both for and against Trump and the Republicans is so concretized that even an event as dramatic as this doesn’t have any material impact on the outcome; or
• The crisis extends through November, causing voter turnout to markedly decline and disproportionately hurting Democrats; or
• Many states (including some critical swing states) respond to the crisis by enabling voting-by-mail in November and future national elections, causing an increase in turnout and disproportionately helping the Democrats; or
• The crisis dissipates in the next few months and has no direct effect in November.

There is, of course, no reasonable way of currently determining the probability of any of these — or other — possible outcomes.

• Democrats “cause themselves to lose.” The number of possible scenarios here is endless. The most obvious involves the Sanders-Warren wing of the party demobilizing because of Biden’s nomination — a possibility heightened if his selected running mate makes them even more unhappy.

But the thought of living another four years with Trump as President will probably trump their unhappiness, motivating most of them to the polls in November and then to the effort of influencing the direction taken by the new Democratic administration.

• 2020 turns into 1984. As mentioned above, Ronald Reagan easily won reelection despite his party losing decisively in the 1982 midterm elections. Could this happen with Trump in 2020? Perhaps, but Reagan had an approval rating of 54% eight months before that November; Trump’s approval is at least nine to ten points lower. He would have to gain some serious approval ground in the months ahead.
• Republicans pull-out all the stops on manipulation, obfuscation, and deception efforts. This is already happening every day. Whether it’s the new Voter ID laws in Kentucky, denying former felons their right to vote in Florida, or purging voter rolls in Georgia, suppression efforts will be pursued with relentless precision. The Republican obfuscation machines are well-oiled and funded. In addition, the Russians have only refined and expanded their misinformation campaigns, Fox News will step-up its reporting on an alternative universe based on alternative facts, and various single-issue groups, from Christian Evangelicals to anti-abortion activists to Israel-first Jews, will redouble their efforts to deflect criticisms of the Trump Administration in defense of their prioritized interests. The possible impact of these collective efforts cannot be overestimated.

November 2020 promises to be a critical moment in American electoral history. While of far less importance, it could also provide another important data point towards our understanding of the predictive power of midterm elections.