Very interesting take on this subject.

I was raised in an Italian neighborhood in Brooklyn and because I look Italian, even though I don't speak it, I had many conversations start in Italian back in the day, only to have to quickly shift gears when I made it clear I wasn't following them. I live in Los Angeles now, where Spanish is a common language. I have watched hundreds of conversations begin in Spanish (between strangers) when neither was wearing a name plate or anything to identify where they were from or what language they spoke. I always assumed it was a "facial recognition" affair as 99% of these conversations zipped along just fine, with the odd ones falling flat with a, Sorry I don't speak Spanish. So, it seems to me that people make instant choices about someone else and dive right in. Most are accurate, but some end up awkwardly as you explained.