Spare us any "not all perverts, uh, I mean, not all glasses!" crying.
Those devices are fundamentally designed to record people without their consent. That is inherent in their design and function -- it isn't possible to get everyone's consent, all of the time, in every public place, in advance. So you are recording others without consent, full stop. And you are doing it on purpose, stupidly and selfishly.
You cannot wear those devices in public without nonconsensual recording others. So everyone SHOULD treat you like a creep and an asshole because if you use such devices YOU ARE ONE.
If you walked around with your phone in hand constantly recording everyone and everything, or back in the day did the same with a camcorder, people would rightly treat you like a freak and a jerk for doing so. That doesn't change just because the camera is now shaped like a fucking pair of glasses.
Your behavior is antisocial. The only problem bigger than your behavior is your inability to recognize your behavior for what it is. Fuck off!