Because it's not apparent. When people talk directly to us, we usually do ok, even with relying on lip reading. As soon as people talk to each other and there is background noise, I cannot follow the conversation. However, since the person then is busy with the conversation they don't notice...