Hi, Chris,
I have slowly progressing sensorineural hearing loss, and have had a history of panic attacks (although they are unrelated to my hearing loss), so I can relate a bit to what you're going through.
Stress - no matter what causes it (a divorce, a health problem, losing a job, moving...