Well, CAPD is not actually a disorder of hearing loss, but a neurological disorder of audiological processing. So, while you will find that your issues and the issues of the deaf will be somewhat different, I don't see any reason for you to be excluded in an open community like the deaf/hoh community.
But, the Deaf Community is something a bit different. Their whole culture is built on the sameness of Deafness and the issues, both social and personal, with being Deaf and using a language other than what the primary culture uses. You might find your entrance there a bit more difficult. But it isn't impossible. The Deaf Community has always welcomed me, and I am hearing. But I would caution you to enter on their terms, not yours. That is why I have been welcomed.