When I initially read stories in the slush pile, I try to be very objective. I look for a protagonist, plot, setting, world-building, and a speculative element. If a story is objectively good I put it in the hold-for-voting pile. The stories in hold-for-voting are ranked in order of preference by each editor. Then we discuss them at the production meeting.
When it comes to the final rankings of each editor, our subjectivity comes into play. I'm not sure if this is a good thing or a bad thing. One editor likes arty stories with a lot of pretty prose and descriptions and a vague or unresolved ending. One editor loves anything to do with dragons. One editor seems to like anything to do with quantum stuff. One editor likes stories with time travel. One editor likes stories with music. (Yes, that's a lot of editors.)
Some people call these preferences reader cookies--because they are a yummy treat for the reader. :)
I guess the bottom line for writers is: if you want to be published in a particular market, you should read the market to see what the editors like. Another take-away is: if your story isn't chosen, it could just be because of the subjectivity of the editors. A different editor or set of editors might love your story.
Next week I'll report on the production meeting.