State and local government politics make building anything in urban California extremely difficult and intentionally expensive.
Voters in California seldom punish their local city or state elected officials for not doing more on housing.
Compare California to states like Texas or Florida, and the difference is massive.
Regulations and bureaucracy are especially irritating in many California zip codes, true, but real estate costs in California respond mostly to the free market. These prices simply wouldn't exist if so many people weren't willing to pay it.
And when you build new housing here, it doesn't bring the other housing costs down, it just gives you more high priced units. Local governments DO mandate a percentage of low-cost housing when signing off on some of the larger residential developments, but these have barely made a dent in affordability. People in the lower paying jobs just have to commute from farther away, where it's still pretty expensive. The people of California do not like this, nor the homelessness issue, but it only stops when people stop paying the ridiculous prices being asked. And they haven't.
California isn't more NIMBY than other states. Most pricy zip codes in America don't want low cost apartments blocking their view, taking their parking spaces, and bringing in lower-income neighbors. In fact California recently introduced SB9, a state law that allows property owners to subdivide their lots and build additional units while bypassing the red tape that used to discourage it. It's an in-fill strategy that encourages adding lower cost units, although not surprisingly some opportunists found a way to game the system.
I don't know a single California home owner who wants to intentionally raise home prices at the expense of the teachers, firemen, small business owners and the grown children we want to live in our communities. I also don't know anyone who wants to move to Texas or Florida. And that's part of it, too.