I don't use iTunes, either, though I do use a Podcatcher. Occasionally there are glitches when server indexes are reset or something (not sure on the details) and it starts to download a whole bunch of old files I already have, but it only happens occasionally and is easily fixed, so I accept it in return for getting my podcasts automatically.
(You people do know that there are other podcatchers besides iTunes, right? I still use good ol' Juice - emphasis on the 'old' - though lately I've been mulling the idea of trying out Miro; I hear it's pretty good.)
I resent having to install iTunes on my computer in order to use their download service. Amazon and other MP3 sellers don't make me do that, so why should iTunes? Bugs me, so I vote with my dollars. On the rare occasions I buy MP3s, it isn't from them.
Plus, my MP3 player is a generic one, the kind you can just attach to the computer and it behaves like a flash drive. No special programs required for getting content on and off the thing, which means iTunes just gets in the way, if it'll let me use the device at all, which I'm not sure of.