I suggest it's used as a pejorative because the concept ruffles the feathers of right wingers who are comfortable in the status quo and don't want the hassle of dealing with human rights and such.
The people you refer to as 'right wingers' created human rights - and this country.