Sorry you had such a rough time.
I don't want to second guess, but my view is this: He's not mad at you. He's mad that other people perceive your deafness as a liability. They see both of you attempting to deal with your own problems as individuals, and perceive that as him not doing enough to help you, and thinking he's a selfish, inconsiderate boyfriend.
I don't think he is, and I think you both know that too. Consider this: he might have been mad as hell, but he was still there when you needed him through the whole ordeal.