I listen to a couple of (non-fiction) casts that come out erraticly, and I agree. I'd rather wait for the good stuff than get a regular feed of sub-par material. I just can't stay with a show that has poor material or really atrocious sound. Fortunately, I have plenty of other good stuff to listen to.
That said, if you can have it both ways, like EA or The Drabblecast, I think it's a huge advantage. Regular content seems to be really effective in building a dedicated following, especially when you're asking for donations on a subscription basis.