No, I do not misunderstand you. It's a fact that some men like having their nipples fondled as much as women do. So I'm saying you should give it a try. ;)
And, no, I am not aware that some boys' breasts swollen during the puberty. Only their private parts (genitals) develop is that I'm aware...