Well, I suppose "fairly" is a tough call and very much open to interpretation but that Submissions Grinder is awesome and exactly what I was looking for. Cheers for that!
Woot! (co-founder of the Submissions Grinder here)
"Fairly" is indeed a tough call, but one you have to make for yourself. One nice thing about dealing with short fiction is that the submissions guidelines generally state the payment terms explicitly in a take-it-or-leave-it fashion. If you have a problem with it, you can choose not to submit. It's not that easy with novel submissions.
Otherwise, there are several factors you can look at. None of these are hard and fast by any means, but are all factors.
1. Longevity. Obviously every market has to start from scratch sometime, so you shouldn't discount something just because it's new. But also keep in mind that in this Internet age it is easy to get a website and to say you are an editor and to get some fiction to put there. It's MUCH less easy to do this consistently over a long period of time. Particularly true of podcasts, which is where the word "podfade" comes in. It's a lot of work to put a quality magazine together, and doing it in audio adds another layer.
2. Website quality. If the website looks like something I slapped together in my junior high computer class in the mid-90s, consider whether you as a reader would want to go there to read rather than something that looks like it was professionally made.
3. Public (and private) correspondence with the editor. If the editor acts like a jerk, either in public or in private, consider whether you want to depend on that guy to work with your stories. I have written off some markets because the editor was very caustic in a semi-private forum for no reasonable cause. If you submit to Pedestal, you get spam as if you had subscribed for a newsletter advertising the editor's writing. Another example is a couple years ago when I saw dozen of genre magazines all pop up which had almost-identical submissions guidelines, including some identical verbiage here and there, but each had a different editor and different focus genre and none of them mentioned the other ones as being sister publications. That smelled fishy, and I stayed away from it.
4. The payment/contract details. Pay rate is certainly a factor. While I think that Highlights Magazine was great when I was a kid, and they pay well, they buy your copyright which means you don't own the rights to your own story every again. I don't submit there.
5. Award nominations/wins. You should never use this as your only judgment of a magazine, but if it wins Hugos and Nebulas, then obviously a large chunk of fandom thinks their work is worthwhile. Lack of awards doesn't mean anything, but presence of them does.
6. Do you like what they publish. If you hate reading everything they publish, then it's very likely they won't like what you write.
7. Submission stats (as previously mentioned).
8. Reputation--this might be hard to determine unless you're on a bunch of writing forums, but for instance, I've heard time and time again that Everyday Fiction is more worthy than their token pay rate might suggest--because they have a very large reader base.
