Just looking at racial demographics, the level of diversity is about the same, it just skews differently. The % white and latino are about the same, but Chicago has as many black folks as San Francisco does asian folks, and vice versa. San Francisco's quite a bit more wealthy, though - median household income upward of $73k vs $47k. Perhaps that contributes to more of a perception of diversity for certain folks.
Chicago's hypercentralized layout may play a part there, too - the downtown area where tourists go is really just for tourists, and to some extent it's particularly for tourists making weekend trips from the suburbs. With the possible exception of Chinatown, all those great neighborhoods you're mentioning are pretty far off the beaten path for a visitor.
> San Francisco's quite a bit more wealthy, though - median household income upward of $73k vs $47k.
That $2k-ish per month is entirely eaten up by the cost of housing. Median rent for a 2BR in SF is $4,000/mo. Chicago is a little over half that. Not to mention much lower taxes than in California.
Chicago apartments are way cheaper than $2k a month. Most of my team pays less than $1k in great neighborhoods such as Lincoln Park and Wicker Park. If you want a building with a pool, doorman, gym, etc. then you can spend $2k a month but you would have to try to find a place that expensive.
To give perspective in Chicago $4k a month can you get a modern 3,000 sqft home or the penthouse in a swanky building vs a studio/ 1br in SF.
Chicago's hypercentralized layout may play a part there, too - the downtown area where tourists go is really just for tourists, and to some extent it's particularly for tourists making weekend trips from the suburbs. With the possible exception of Chinatown, all those great neighborhoods you're mentioning are pretty far off the beaten path for a visitor.