No way to know for certain if this is due to your deafness. Cliques happen in every group. But assuming it is and assuming that the hearing world is something you want to be a part of, there are a few things you could do.
1) First consider that perhaps your co-workers are just trying to avoid...