Ever-changing population predictions frustrate Bay Area smart-growth planning


The stop-and-go flow of the toll creates early morning traffic for commuters into San Francisco. Photo by Tearsa Joy Hammock / SF Public Press.

State, regional forecasts vary widely, generating uncertainty about long-term housing needs

State and regional planning agencies have produced differing predictions of how many people will migrate to the Bay Area in coming decades. The disagreement is frustrating efforts to forge a consensus on how many hundreds of thousands of new homes to build across the region, and where.

In May, the California Department of Finance took a fresh look at economic forecasts and officially backed away from its prediction that 9.5 million people would live in the Bay Area in 2040. The state now says it is likely to be closer to 8.4 million.

But the Association of Bay Area Governments pegged the population for the same 2040 target date at a robust 9.3 million. The agency is charged with developing Plan Bay Area, an ambitious agenda to reshape the sprawling region by building 660,000 new homes in the urban image of walkable, transit-friendly San Francisco.

Population prediction is an inexact science, and previous regional forecasts that relied on overly optimistic job-growth estimates proved to be wildly wrong — some by more than 50 percent. Planners say they are doing the best they can with the mathematical tools they have and are constantly adjusting the numbers as the economic reality changes.

The complexity of these calculations has created tension within the regional association itself — between board members and professional planners, who break down the numbers and assign growth to specific cities. The group’s elected chairman, Mark Luce, confessed in a recent interview that to him, “the numbers just aren’t making sense at all.”

At stake in the population debate is the credibility of the Bay Area’s “smart growth” vision — compact, energy-efficient communities that encourage people to spend less time in their cars.

The move is a key component of the state’s goal to cut California’s per capita greenhouse gas emissions by 15 percent in a little more than two decades.

Planning agencies, led by the Association of Bay Area Governments, seek to build enough housing to prevent the expected population explosion from creating an ecological and sociological disaster zone.

While state demographers see relatively meager economic growth for years to come, the regional association’s assessment assumes that an improving job market will attract 1.2 million more jobs in a little less than three decades. Right now there are about 3.4 million jobs in a region of 7.2 million people.

While demographics play a role in the predictions, current trends in the real estate market also affect their models.

By some measures, the region’s housing market is rebounding. Numbers released June 5 by the online real estate directory Trulia show increasing demand for rentals across the Bay Area and especially in San Francisco. Over the previous year, the average rental asking price in the city is up 14.4 percent, the sharpest increase in cities among the nation’s top 25 rental markets. Oakland and San Jose saw rent increases of 11.4 percent and 9.1 percent, respectively.

Nonetheless, critics say the regional agency’s projections for both population growth and the resulting housing needs are way too high. The uncertainty has spurred objections from the region’s 110 cities and counties.

Many of these objections come from relatively wealthy communities that have long resisted providing the expected level of low- and moderate-income housing to house their lower-wage workforce.

But some argue that they would waste resources by rezoning their neighborhoods for more development than they need.

“Are they creating problems for the state by having such a high expected growth rate, and then using that to encourage more house building than might be appropriate?” asked Greg Schmid, a city councilman in Palo Alto. He is an economist and consultant for corporations and the city of San Francisco and a former Federal Reserve researcher.

Schmid questions the economic assumptions behind the regional agency’s population forecasts, citing recent examples in which some of the smartest professional planners turned out to be dead wrong when actual census figures were published at the start of each decade.


The Association of Bay Area Governments has not ignored the criticism from Palo Alto and more than a dozen other city and county governments.

The current plan, announced in March, is to add 660,000 new homes by 2040. But that was a radical revision from the previous estimate of 903,000 homes. Planners now acknowledge that they had not sufficiently accounted for the hundreds of thousands of suburban houses seized by banks or abandoned by owners in the foreclosure crisis that began in 2008.

The loss of credibility for Plan Bay Area could add to criticism that the whole endeavor is not worthwhile.

Planners say that underestimating housing needs could lead to more unplanned housing sprawl, worsening the region’s carbon footprint.

But overestimating could also have economic consequences for the 200 planned “priority development areas” in about 60 communities if it causes them to make bad infrastructure investments.

If Plan Bay Area were to fall short, the state would need to re-examine its climate-change strategy, which relies on regionwide land-use changes to reduce carbon emissions from cars.

Planners acknowledge that shifting economic realities might undercut their cause. The current plan, as ambitious as it is, reaches only 9 percent greenhouse gas savings per capita, according to regional agencies’ own estimates, and there is no clear path to reaching the state’s goal of 15 percent.


Overestimating population growth is a common complaint, planners say. And as the fluctuating numbers show, California has been anything but clairvoyant.

The state finance department, for instance, has consistently overestimated growth in California for the past two decades. The department’s projections for 2000–2010 overshot Bay Area growth rates by about 56 percent. (The prediction error for the entire state’s growth was 48 percent.)

Private think tanks and universities also make off-the-mark population predictions. In 2005, the Public Policy Institute of California compiled a five-year “consensus” forecast that aggregated the work of academic and government researchers. That figure turned out later to be 50 percent higher than the state’s actual population growth.

The component studies — from the Department of Finance, UC Berkeley’s planning school, the UCLA Anderson Forecast and others — on average forecast a 15 percent population increase. But the actual growth, the 2010 census figures showed, was 10 percent.

“Population projections for California are especially difficult,” the institute noted in a report. “In addition to overweighting contemporary trends, forecasters are notoriously bad at predicting fundamental demographic shifts.”

One piece of advice from the Public Policy Institute: “Planners should consider alternative population scenarios.”

Part of the problem with California is that it has historically had a “fairly volatile” growth pattern with a lot of in- and out-migration, said Hans Johnson, the Public Policy Institute’s research director. The point of the report, he said “was to say that it’s dangerous to put all your eggs in one basket with a set of projections because they are uncertain. It’s best to try to make your plans at least flexible enough to accommodate alternative future scenarios.”

While Plan Bay Area relies on one set of projections for now, plans will be updated every few years.

This can be done, he said, by presenting a range of high, medium and low projections. Plans should then be based on the average and there should be contingency plans “that would allow you to accommodate potential changes.”

But California does not routinely produce alternative scenarios, he said. Those have to be generated by regional agencies themselves, including the Association of Bay Area Governments.

“Recognizing the uncertainty of projections is something that makes planning very hard,” Johnson said. “It’s a tremendous challenge for ABAG and for people who are trying to plan housing.”


Some critics of Plan Bay Area, especially among officials in cities that will have to plan for significant growth, see a pattern of overly aggressive growth predictions.

Palo Alto has officially questioned Plan Bay Area’s methodology, while protesting the designation of a portion of its downtown and a stretch of El Camino Real as priority development areas. The plan currently requires the city to make room for 7,130 new housing units by 2040.

Officials from Contra Costa, Marin and other Bay Area counties are also skeptical about the numbers.

The Contra Costa County Transportation Authority, one of the regional agencies protesting the plan, has complained in writing that projections “remain at the high end of remotely plausible outcomes for the forecast period.”

The Marin County Board of Supervisors has called for the projections to be peer reviewed.

After the Marin city of Corte Madera criticized figures in the plan’s initial vision scenario, the association cut the housing projection by nearly 50 percent, to 270 units. Yet the city’s leaders voted unanimously in March to pull out of the association because they were still skeptical.

Other citizens’ groups, including the Novato Community Alliance and Save Orinda, have also expressed misgivings about the numbers.

“As they make their projection into the future, the numbers seem consistently higher than the trends that have been established in the Bay Area over the last two decades,” said Schmid, the Palo Alto councilman.

“I think the new Department of Finance numbers are more practical and reasonable,” he said. “ABAG should let communities decide whether they want to be aspirational or practical. That’s just good policy when facing the uncertainties of the future.”


Luce, of the Association of Bay Area Governments, is clearly frustrated by the political tumult around the growth predictions. “Every time we come up with these numbers, there’s a lot of heartburn,” he said. “It’s the least favorite activity ABAG goes through.”

As the regional land use planning agency for the nine-county San Francisco Bay Area, the association must produce a “regional housing needs allocation” every eight years, assigning the number of housing units, in different levels of affordability that, each community must plan for.

Determining the total housing need for the region is the job of the California Department of Housing and Community Development. Luce said his agency then merely tries to sensibly spread the burden of that growth across the region.

Here’s how it works: The state’s housing agency looks at the finance department’s population projections, checks the current housing stock and calculates how many more homes California will need over eight years. Then it divvies them up across the state based on the population predictions. Regional agencies can ask for those numbers to be revised, though state housing officials have final say.

“So ABAG is in the middle,” said Luce, who is also a Napa County supervisor. The association’s board consists of elected officials from Bay Area cities and counties.

“It’s a state law — you’re going to get an allocation whether you’re part of ABAG or not,” Luce said. “But with 101 cities and nine counties, somebody is not going to be happy with what you’re trying to do.”

Regional planning is hard enough, but to further complicate things, it must now meet state environmental targets for transportation efficiency and sustainable growth. Plan Bay Area is designed to comply with a 2008 state law, Senate Bill 375, which calls for steep reductions California’s transportation-related greenhouse gas emissions from cars.

Under the law, every region needs its own “sustainable communities strategy” to put new housing close to public transportation. In the Bay Area the task falls to Luce’s agency, as well as the Metropolitan Transportation Commission, the Bay Area Air Quality Management District and the Bay Conservation and Development Commission.

The plan aims to expand housing not just to accommodate a growing population but also to use housing growth as a tool to “create jobs to maintain and expand” the regional economy.

That sounds backward, said Schmid, the Palo Alto councilman: “ABAG’s job is to distribute housing mandates, not allocate jobs.”

While Luce defends the regional planning agency he leads, he also is fighting for his own Napa County turf. He said he was particularly upset with the housing allocation of 5,640 units for the largely rural county, though he expected the agency eventually to lower that number.

Giving out housing targets to cities and counties, he said, is a thankless task.

“Sometimes what happens is you get in this rut of, ‘OK, the state has given us an allocation, you got to distribute it,’” he said, “and lose sight of what we’re really trying to do. That’s where you have to step back and hope that there was some planning in this whole process.”

That kind of talk hints at deep divisions between the professional staff at the Association of Bay Area Governments and its directors, many of whom appear to be dubious that what they are doing adds up to a coherent regional strategy.

The agency’s planning director, Ken Kirkey, defended the staff’s efforts. “What we do is long-range planning,” he said. “Thinking forward to 2040, you can’t be stuck in the mindset of 2012, which is really hard.”


Some experts in the field of population prediction said the concern with quantifying future growth is overblown.

Housing targets for as far off as 2040 are hardly something to get worked up about, said economist Stephen Levy, director of the Center for Continuing Study of the California Economy, a research group in Palo Alto. Levy has been helping the Bay Area planning agencies tackle the job growth calculations based on national and regional economic trends.

“These projections are long term in nature and have a self-correcting mechanism in place,” he said. “If trends change, the growth projections will change. But right now there’s every indication that ABAG’s projections are low. We used fairly modest numbers,” he said.

Other urban planners maintain that the population projections are not a problem because to a certain extent the region can influence the size and shape of its own growth by setting a target.

“What’s a reasonable number?” asked Egon Terplan, the regional planning director at the San Francisco Planning and Urban Research Association. “Nobody has a crystal ball for the future. It’s easy to be skeptical of any number that comes up. I would be skeptical of someone who says these numbers don’t feel right.”

Terplan said that while job growth may have slowed in recent decades, the Bay Area was in “a pretty good place economically” and would continue to add population.

“The main future impediments to that growth are, frankly, land use policies that are restrictive of growth, and particularly the ability or inability to add significant amounts of housing,” he said. “I think that what’s important is having a number that encourages places to plan.”

Terplan said he hoped that higher population growth projections would end up encouraging economic growth. But if the region sets housing needs too low, he warned, communities would not make the big planning and zoning adjustments they would need when growth returned. They would have no impetus to prepare for the eventual economic boom.

“Let’s just think about the long-term goal for a moment, getting back to what are we trying to do,” he said. “It’s not producing a report. It’s not a fight about numbers on a piece of paper. It’s to change the way we do business.”

Don't miss out on our newest articles, episodes and events!
Sign up for our newsletter

Read full coverage of Bay Area smart growth in the San Francisco Public Press Summer 2012 print edition, on sale at retail outlets around San Francisco and online.