> I don't believe this is the case as since 2008 the U.S. construction industry has not even built enough units to keep up with population growth. I believe this is a supply problem which is two-fold.
You certainly have valid points, especially in San Francisco. Nevertheless, looking at the broader market, outside of California, the market appears to be working: builders find a way to build what will sell and maximize their profits. Last week, I drove around a planned community in a middle-America state. Developers started building houses around ~2010 and are still building out the area. These developers had no problem building thousands of homes that were selling well until very recently.
I'm glad the builders in your area have been able to work consistently. I would guess that they are likely riding a trend of a decreasing number of builders who are increasingly professionalized and technologized their operations. Regardless, that is an anecdote and the statistics paint a pretty clear picture.
A counterargument to this is that population growth has not been very high in the past few decades: https://fred.stlouisfed.org/graph/?g=SUL3
You certainly have valid points, especially in San Francisco. Nevertheless, looking at the broader market, outside of California, the market appears to be working: builders find a way to build what will sell and maximize their profits. Last week, I drove around a planned community in a middle-America state. Developers started building houses around ~2010 and are still building out the area. These developers had no problem building thousands of homes that were selling well until very recently.