I find the above statement to be not entirely true. For one, not only have I read stories in which the "you" is an actual character within the story and not the reader, but I have written a story that way myself (a story that had even people who claimed to hate second person saying they enjoyed thoroughly, fyi). The "you" narrative needn't be directed to the reader, though many people seem to think that is the only way second person can be used--which isn't true. I think the "you" being directed at someone other than the reader makes second person easier to swallow.
Second person is, after all, just another PoV, or perspective depending on which term you prefer. (I use PoV, though that is probably wrong, it's how my brain clarifies first, second and third.) Just like first and third, when written well, it works. And a story must be told in the best way it can be. If second person is how it works, then I'd shame anyone who changed to first or third simply because second person has some strange stigma attached to it. When I wrote my second person frame-story, one beta reader suggested I change it to third. I did. It sucked. Not only by my standards, but by all the other betas too. First person didn't work either. Second person was the only way the story felt right. It lost something when that perspective changed.
In my opinion, use what tools you need to write your story as it should be written. Use only what you need. If something is used as a gimmick, as second person almost always is, then figure a way to make the story interesting without the gimmick. If the story needs it, then use it. The gimmickry is what makes second person less desirable than the other perspectives. I rarely see a second person narrative that doesn't scream gimmick.