Curious, when did we ever buy a car direct from a manufacturer? I'm 43 years old and I don't remember a time when auto manufacturers offered direct sales? Didn't Saturn try to do this, I think? I did European Delivery in 2004 from BMW (which was killer, BTW) and that felt like a direct sale even though the dealer pushed all the paperwork on my behalf for roughly $600 over invoice.
Retail distribution has been a staple of the American (and world-wide) economy. In order for these manufacturers to reach everyone, they need to scale thru retail distribution. What gets me the most with these markups is that there is no "brand protection" from these auto manufacturers. Essentially, they create this grey market that is widely acceptable because of the "limited" status of these specialty cars. It fosters greed and creates that cycle of "resentful customer > slimy dealer > resentful customer > slimy dealer". I actually know of a bunch of guys who sell cars who do have a moral compass but it's these antics as you stated above that just add fuel to the fire, imo.