Larry Summers’ provocative address at the International Monetary Fund should serve as a dire wake-up call to policymakers and pundits.

The U.S. economy is not recovering. In fact, recovery from this depression cannot be taken for granted or simply assumed to be imminent.

Summers’ prognosis has profound implications, threatening to turn upside-down macroeconomic thinking of recent decades and conventional wisdom regarding fiscal prudence. His message is also difficult to square with much of today’s political grandstanding, policy proposals, journalism and opinion writing intertwined in economic debates.

The Overlooked Unemployment Problem

The unemployment rate in particular seems to be a major source of confusion, contributing a misplaced sense of complacency about the direction of the economy.

A recent Washington Post article by the normally stellar Eugene Robinson optimistically pointed to unemployment on a “well-established downward trend” and a better-than-expected gross domestic product (GDP) report, falling budget deficits and stock indices at record highs.

Yes, the unemployment rate is on a clear downward trajectory. But it’s falling for problematic reasons, and the beleaguered labor market is not recovering.

Throughout the post-World War II era, a falling unemployment rate meant a higher share of the population was employed. This relationship is fundamentally broken in the aftermath of the Great Recession, as the figure below depicts.

Summers’ reflection that the economy isn’t recovering started with the grim reality that the share of working adults hasn’t budged from severely depressed levels over the last four years.

The Unstable Labor Force

The recent disconnect between the unemployment rate and employment-to-population ratio is quite simple.

In order to be classified as unemployed, someone without a job must have actively looked for work in the previous four weeks. The unemployment rate is the ratio of unemployed workers to the labor force—the sum of employed and unemployed workers.

The unemployment rate has been falling overwhelmingly because discouraged and unemployed would-be workers are dropping out of the labor force.

Much of the of employment growth in monthly reports—roughly 100,000 jobs a month—is essentially absorbed by a growing population putting countervailing upward pressure on the labor force. Hence the stagnant share of employed persons as a share of the population.

The labor force participation rate reinforces this reality. After holding roughly steady in the five years ahead of the recession, the civilian labor force participation rate began a consistent descent in late 2008. As of October 2013, labor force participation fell to a 35-year low.

So, why are people dropping out of the labor force?

A gradual decline in labor force participation would be expected as Baby Boomers begin to retire. Additionally, a rising share of the working-age population is enrolled in school and thus not counted in the labor force.

But the overwhelming reason for the sharp drop is exactly what Summers’ speech was getting to: inadequate demand for output and workers on a scale unmatched by any economic shock since the Great Depression.

Unlike other post-war recessions, policy responses by the Federal Reserve, tasked with stabilizing both unemployment and inflation, are incapable of alleviating this collapse in demand.

Fundamentally, this isn’t a story about skills mis-match, robots or any other such “structural” scapegoat. The simplest debunking of such structural claims is that the unemployment rate, despite being rather misleading in aggregate, has roughly doubled proportionately across every measure of educational attainment.

The Missing Workers

Heidi Shierholz, a labor economist with the Economic Policy Institute, estimates there are roughly 6.1 million workers “missing” from the labor force because of economic weakness. The unemployment rate would currently register 10.8 percent instead of 7.3 percent if these missing workers were still counted in the labor force.

Neither employment nor aggregate demand has recovered anywhere close to potential—and the labor market and GDP are not currently on a trajectory for meaningful recovery.

As I’ve explained, there’s a critical but nuanced distinction between economic expansions versus economic recovery. Economic growth is necessary for recovery, but it isn’t sufficient for recovery.

The stock market and corporate profits have rebounded nicely for those heavily invested in equities. But, we’ve seen an incredibly disparate rebound for capital and non-recovery of labor coming out of the Great Recession—one that’s fueling income inequality growth, as I’ve argued in the Fiscal Times.

The budget deficit has fallen markedly, but it’s impeding recovery and is the prime cause of the deceleration in growth.

The “fiscal cliff” crisis that had both sides of the aisle espousing concern about renewed recession was simply a problem of budget deficits closing too quickly. Congress ignored the cliff’s teachable moment and left the more damaging half of the scheduled budget restraint to materialize.

This time really is different, in a bad way. Policies reasonable at full employment are utterly irresponsible and counterproductive now. Economic indicators suggesting economic health improvement can be grossly misleading. Indeed the unemployment rate could start rising sharply if the labor market started going gangbusters and would-be workers re-entered the labor market.

So don’t let yourself be fooled by the drop in the unemployment rate—the economy is stagnating and will be for the long-haul. Unless policymakers reverse the emphasis on deficit reduction and prioritize genuine economic recovery.