Hi Uuri,
Welcome to BP! I'm also a San Francisco resident living in Hayes Valley. The real estate market in SF is quite unique to say the least. It reflects the economic principle of supply and demand. With only about 7 miles width and depth, coupled with tight city coding, the city does not have enough units to satisfy demand to live here.
In addition, the social media bubble on top of explosive growth from companies such as Apple and Google has created a situation where many newly wealthy holders of stock options in said concerns are able to bid up the price of properties beyond long term value.
I could also mention the extreme bias towards tenants that exists in San Francisco. Suffice it to say, I'd advise you to become well read on local laws and ordinances that impact landlords if you choose to let out your unit. I recommend consulting the SF Tenants Union website. That organization publishes a manual (written for the benefit of the tenant) that every landlord should read.
This conditions have caused me to be a resident (and property owner) in SF, but to make my real estate investments elsewhere.
Lenders in San Francisco are, as you note, very hesitant to lend except to those who have a solid work history. It can be very difficult when you're an entrepreneur or work for a start up that does not have well established funding. Remember too that you'll be competing against many cash buyers who've just traded in their Zynga, Facebook, etc., options.
I looked into acquiring a SFR in the Bay Area earlier this year, but was overwhelmed by the amount of all cash offers (over asking) I had to compete with. In the end, I decided to: 1) keep my condo, 2) improve it, and 3) invest in real estate in other markets to create alt cash flow streams.
It is all enough to make one's head spin...In any case, SF has a ton to offer. I wish you success whatever path you decide on.