I recently finished a stint as one of several first-readers for a literary journal's fiction contest. I had a stack of 50 stories to read, which I had to whittle down to a top five that were sent on to the final judge(s). This was my first time on the other side of a literary journal, reading submissions instead of writing them, and though I was initially alarmed at what I foresaw as an agonizing slog of painful decision-making, it turned out to be easier than I thought.
There were a number of stories I set aside right at the beginning:
Genre stories: types of stories not published by the magazine in question, e.g. fantasy, horror, science fiction ---- I was surprised by how many of these there were. Probably at least a fifth of the pile I had, maybe more. Many of these were well-written, but didn't establish any kind of specialness outside of the traditions they were working in.
Stories with almost nothing at stake, e.g. twenty pages of how to get a kid to eat his vegetables. Ones that were boring from the first page. Again, many were technically well-written.
I was also surprised by the number of stories about someone on his/her deathbed, looking back on a life (a potential variation of nothing at stake, in a way, if there's no present conflict or narrative line through the reminiscences) or set in 50s-style small towns (nothing wrong with this -- I was just taken aback by how many there were).
At the end of this process of removal, my pile of still-viable stories was significantly smaller, and finally there were a few hard decisions, after all. But not as hard as I expected.
I don't know if I'll ever get the opportunity to try this experiment, but I think I could have sorted the stories into three piles (Yes, Maybe, and No) based on the titles and first sentences alone ---- and the piles would not have looked very different from the ones I ended up with after reading all of the stories. Well, perhaps the Maybe pile would be bigger if my selections were based on just the first sentence. But the No pile would be the same, I'm sure of it.
All in all, it reminded me that the short story is a difficult animal. It is not enough to be a good writer to win a story contest. You need to read a lot of short stories and try to understand how they work. And it REALLY helps if you read previous issues of whatever magazine you're trying to publish in. (Until this experience, I could never understand why so many writing resources harped on this point.)
In the course of reading the submissions, I happened upon this blog post on How Not To Write a Story (written by a writer judging a story contest) and it was an eerie echo of many of the things I saw. Good advice.
The most exciting part of the process, though, was finding the gems and being surprised by where they took me. All the best stories were surprising.