Hearing people can't help but overhear other people, whereas signers (deaf or hearing) can be polite and look away from other people's conversations.
In our ITP we were told it is polite to inform people in a situation like Audiofuzzy's that you can sign, so their conversation may not be as private as they think. I think our teacher's motivation in telling us this was to encourage us to interact with deaf people, but personally I've never felt comfortable with this. I just try not to look at them. If I notice a break in the conversation I may sign something like "nice day" or other small talk just to give them a hint that there's a signer around, but generally I try to ignore it, although it can be difficult. (Not because I care about the gossip of strangers but because it's just hard for me to look away from signers!)
What positive results are you expecting to accomplish with a comment like that, I wonder?