Then you should communicate that better. “aren’t these just autistic people with aspergers” makes it seem like you think everyone with aspergers is like this.

I have a friend with Aspergers who would be quite offended by that comment. These people are just idiots, don’t blame it on a diagnosis.