As a transperson (MtF binary transsexual), I'm acutely aware of public attitudes toward the transgender community.
Even among the LGBT community and among some groups of radical feminists- who should be our natural allies- there is a lot of ignorance, prejudice, and confusion over just what the term "transgender" means and what the trans community wants as far as recognition.
I apologize in advance as I am still trying to get a grasp on non-binary gender identities and how our society might better accommodate them; my perspective is largely trans-binary and I do not mean to maliciously exclude my non-binary friends.
More behind the orangey cumulus cloud...
1. Gender dysphoria is a delusional disorder related to schizophrenia
More often than not, this is a line bleated out by guys with neck beards and trilby hats, and encouraged by quack psychiatrists like Toronto's Dr. Joseph Berger, a proponent of damaging ex-gay therapies. But not to be outdone by the neckbeards, at least one radfem blog that typically misgenders transgender women as male has cited this misinformation.
This is so out of touch with the medical reality of gender dysphoria that it's laughable, but it isn't being called out near as much as it should be.
The fact is, for many decades, the medical community believed exactly this: that identifying as something other than one's assigned gender was akin to a delusional disorder. And following up on this hypothesis, they did attempt to treat gender dysphoria with coktails of antipsychotic and antidepressant medications... to no avail.
In fact, many people seeking cross-gender hormones and surgery are on psychiatric medications. Even I take meds for anxiety. These medications may ease some of the emotional problems caused by having to live with a body and brain that don't match, but they don't fix the underlying discomfort.
2. Many transsexuals regret their surgeries
A number of high-profile cases have come to light over the last few decades in which someone who was convinced that surgery was the right option for them has publicly denounced the procedure as a mistake. Figures such as Walt Heyer (whose website. "sex change regret," is very coy about the fact that he is promoting a conservative Christian agenda) have become a cause celebre for those who want to discredit transsexualism as the product of "political correctness."
The truth is much more complex. As this page explains, there is usually a pattern to those who regret their decision to get surgery. It is usually long-term intense cross-dressers who are transitioning for a sexual thrill and who want to be the center of attention, or have unrealistic expectations, who usually end up regretting their decisions. They typically have a poor response to hormone therapy (complaints about low libido are the norm), and typically transition late in life (after about age 40).
This isn't to say that everyone who fits this profile in part is going to make a mistake, and the concept of "autogynephilia" is largely debunked (PDF here), but there are people who definitely should not get surgery and do it anyway. They do not, however, represent all or even a majority of transsexuals!
The WPATH Standards of Care exist to help sort through people who may get surgery for the wrong reasons. That being said, the standards are only guidelines, and those who should probably not get hormones and surgery can feed their therapists the "correct" answers to get treatments that may not be the best course of action for them; on the other hand, inexperienced or incompetent therapists can easily set the thresholds too high for people who really do need hormones and surgery. For this reason, many clinics now operate on an "informed consent" model that focuses on cultivating realistic expectations and intensive guidance during treatment rather than setting high thresholds.
3. Someone can gain access to women's facilities simply by declaring that they are a woman
It is a staple of right-wing scaremongering: an obviously male individual claims to be transgender with the ulterior motive of gaining access to women's restrooms and locker rooms. Noted transphobe and world-class creeper Mike Huckabee famously made the worrying statement that he would have done just that had the option been available to him.
But the fact is, it's not that easy to gain recognition as a trans person. In fact, it's too damned hard! We go through lengthy batteries of psychiatric evaluations, sessions with physicians and counselors, and miles of red tape to even start our transitions. And I promise you, some neck-bearded pervert who tries to use a women's restroom or locker room will not get a kind reception, even in a liberal state like Oregon where a revised ID isn't required to use women's facilities.
I do think that gender-neutral facilities are a good idea, but they're not a cure-all. I think that in the case of single-use restrooms, gender neutral is the way to go. But in the case of multiple stalls, the last thing we want is to be forced to use a stigmatizing third restroom for "other" by way of some "compromise." I come from a part of the country where many of the older gas stations still have three restroom doors, and although the old "colored" restrooms are now mostly bricked up or converted to storage, that third door or some remnant of it is still there, a harsh reminder that separate but equal doesn't work.
4. Gender is just a construct, so "transgender" doesn't really exist.
While it may be true that less rigid gender roles might ease the discomfort of some people, particularly those who identify outside the gender binary, it isn't that simple.
While this is not true for the entire trans community, a large number report being drawn to stereotypically "feminine" or "masculine" things from a young age.
Hormone therapy is widely held by those of us who have taken it to change emotional responses (in my case, in a healthier direction that works better with my brain structure). The jury is out in the medical community but the common experiences of thousands of transfolk don't lie.
Furthermore, gender roles and what they imply differ across cultures, but gender variance occurs across the human species.
It is my informed opinion as a social scientist that while gender has a large social component, there is some biological basis to gender and that the phenomenon is so poorly understood that pronouncing that gender is "just a construct" is reductionist.
Furthermore, even if one takes the element of gender out of the picture, there is still the element of transsexualism which is both apart from and inextricably linked to transgenderism. A transsexual, by definition, is someone who seeks to physically alter their body by hormones, surgery, or both. It is my belief that removing gender roles will not eliminate the biological reality of sex or the need for those of us who are uncomfortable with our physical sex to fix that disconnect.
So no, transgenderism and transsexuality can't simply be wished away by eliminating gender categories (assuming we could eliminate them in the first place). To say otherwise is presumptuous and ignores the broader current of human history and the bulk of biology and social science.