OK. I take that back. It is true on a literal level.
However, when we're talking about intellectual or experimental writing, most authors know that their work will only be understood by a slice of the reading population. In order to be understood by more people, they would have to alter their writing goals. So while I think it's true they wish more people appreciated experimental writing (see Ben Marcus, Harper's), they don't want to go to the audience, they want the audience to come to them.
Sometimes obscurity becomes cachet. I'm taking a writing workshop right now with a woman who won the Pulitzer and is reputed to be a genius. Recently, a bunch of us MFA muddlers were completely flummoxed by a very beautiful but puzzling piece written by another student. Our professor declared it perfect in its thematically linked, non-linear, non-explicated splendor. There's a certain prestige in having your writing be understood by the genius.
It's the opposite of Greg Bear's "get their beer money" mindset.