If you keep your vehicle locked up, it should be fine.
If your vehicle is broken into,
if the remote is stolen,
if the crook gets your home address before you notice the break in,
if you don't disable your garage door before the crook gets there,
if you don't keep the door between your garage and house locked when you're not home, then it could be a problem. But that's a lot of "ifs".
We've had remote controls in several different vehicles that are parked outside at public places for over 17 years, and have never had a problem.