Network effects are a (nonlinear) function of network size. The bay area is limited by local zoning ordinances and can't scale much larger. There's only so many locals to displace and new construction happens at a snail's pace. New York has a much larger scale already, so at this rate it's just a matter of time until New York overcomes the network effects that the bay area currently enjoys.
The factors that could keep the bay area dominant are:
(1) Rapid densification and development of the region pushing down prices and attracting newcomers (not likely)
(2) A tech recession halts the expansion of the sector and potentially begins a phase of consolidation
What I want to know is how much this extends into real engineering jobs, not just plumbing and web apps. I totally buy that ops and legal jobs in tech companies for example are growing strongly in NYC, but when it comes to real engineering problems I'm still only seeing a few teams at a few companies in NYC.
Don't get me wrong. I want this to be true, because I want Silicon Valley to lose its monopoly on the best engineering problems, but it's when the most interesting problems move out of Silicon Valley will I buy the idea that it has been dethroned.
The factors that could keep the bay area dominant are:
(1) Rapid densification and development of the region pushing down prices and attracting newcomers (not likely)
(2) A tech recession halts the expansion of the sector and potentially begins a phase of consolidation
(3) A paradigm shift towards remote work