29 July 2023

Here Come da Judge


 

True story: While I was trying to figure out what to post for today, I was asked by a writer friend to serve as a judge for an upcoming fiction competition. This kind of thing would probably be nothing new for you, and wasn't for me either--I've judged dozens of fiction-writing contests over the years. (That says nothing about my qualifications; it's just something that happens when you've been around and writing for a long time. In any case, I was honored to be asked.)

I'm sure you know the types of contests I'm referring to. Some are local, some regional, some have solo judges, some are judged by committee, some have cash prizes, some are sponsored by groups or conferences that have the winning stories appear in an anthology. Arguably the most prestigious competitions (certainly for mystery writing) are those for national awards like the Edgar, Shamus, etc.

Anyhow--long story short--since opportunity has knocked, I figured I 'd use that for today's post.

To me, judging writing contests is a mix of fun and work. Fun because some of the entries you have to evaluate turn out to be great stories; work because most of them don't. But I assure you I've learned a lot about writing from each of these endeavors, and I've also learned quite a bit about what I suspect editors, agents, and publishers have to go through every day in the process of selecting which stories/novels to publish.

An example, and some observations:

Assume you have been asked to be a judge, and you find that you'll have a hundred short-story manuscripts to consider, and your task is to pick the best three.

When your stack of entries arrives, I predict that about a fourth of them, maybe a fifth, will turn out to be good, well-written stories. That's just usually the way it happens. Also, another fourth of the stack will be terrible stories. Those that are left--about half--will usually be somewhere in between. I realize that's a big generality and that there's nothing certain about what you'll find in any set of manuscripts to be judged, but so far I've found that the old 25-50-25 percent division is pretty close. Strange but true.

Another observation: whether you're one of a group of judges or if you're doing it all yourself, you'll probably find that your first read-through of the stories is to weed out the bad ones. That sounds like a negative way to approach all this, but it's natural, and is pretty much the way editors do it. If/when you find things in a story that just don't work at all, that story goes in the reject stack and you move on to the next one. The stories that are left when you're done are the ones that'll be re-considered. (This, by the way, is the whole premise of Noah Lukeman's excellent book The First Five Pages. It says that a publisher/agent/etc. can usually decide in the first five pages of a novel manuscript whether to reject it. For short stories, it's obviously a much shorter span--maybe the first page or two, or even the opening paragraphs.

Once the rejected manuscripts are put aside, you'll probably then re-read the others and do the same thing all over again, this time comparing them with each other in terms of quality. Again, I predict you'll end up with anything from fifteen to twenty-five out of a hundred that are truly good stories, and then you'll have to decide which of those are the very best.

One thing that I find difficult is when the contest organizers require you to fill out a detailed scoresheet evaluating different parts of each story, assigning points to things like plot, characterization, dialog, setting, viewpoint, and theme, and coming up with an overall total to determine the winners. I'll do that if I'm forced to, but I think it's unnecessary work. Good stories don't always hit the normal checkboxes. Some of the best stories I've ever read do strange and unusual things with plot, POV, and so forth--you know what I mean. I prefer contests that allow the judges, solo or teamed, to come up with which stories they think are deserving of the top honors without resorting to the detailed "Fiction Writing 101" lists and rules and checkboxes. But that's just me.

I also don't like it when contest organizers tell me I must read every story all the way to its end. That's a terrible waste of time. If you're going to trust me enough to be a judge, trust me enough to know when to reject a story, and--as mentioned earlier--that decision might happen early in its reading. 

As for whether the judging is "blind"--some contests withhold the authors' names--that precaution honestly doesn't make any difference to me. Some of the best stories I've seen have come from writers whose names I didn't know at the time, and some stories by known authors have disappointed me. As it turns out, the upcoming competition I mentioned will feature blind entries, which is of course an effort to assure entrants of its fairness. But I think it rarely matters to a judge.

NOTE: One thing I try not to do (although I have, when I didn't know it at the time) is serve as a judge for a competition that requires entrants to pay fees. I don't agree with that practice and I don't enter those contests, just as I don't submit stories to markets that charge submission fees. 

Questions:

Do you often participate in the judging of writing competitions (big or small)? Have you ever done so? Did you enjoy the experience? Did you learn anything from it? Are there any past judging gigs that were particularly fun or interesting for you? Did you have a set routine by which your evaluations were made? If a team effort, what did you think of working with other judges? How about the scoring process? Did you find it overly restrictive, or were you given free rein?

I've already mentioned that this kind of request (to be a judge) was nothing new. Well, neither is the fact that I said yes. When the person asking is a friend, it's hard to say no.


I'm hoping I'll find some great stories.



47 comments:

  1. Interesting piece, as usual. I have never judged a contest, as far as I remember, although I have certainly judged awards. You mentioned several of the points I would have brought up: I loathe being asked to fill out forms (especially on flash fiction!). If pressed I can give a letter or number grade to each story but I would much rather just say, these are the best five, ten whatever. I can rank them if need be, but if there are several judges my preference is for all the judges to list their top (say) five and see if several stories rise to the top. Second, I absolutely agree that you don't need to read the whole story (exception being if the writers have been promised critiquing of their stories, in which case I assume I am being paid?). As some editor famously said "You don't have to eat the whole egg to know it's rotten."

    ReplyDelete
    Replies
    1. Rob, that was a poor word choice, on my part. I was referring to judging for all competitions--awards as well as contests. And yes, I'm excluding the possible *critiquing* of the entrants' stories. If critiques are part of the deal, I wouldn't mind filling out the checklists, etc.

      For straight judging of stoires, I don't even see the sense in assigning number or letter grades. Like you, I will provide those when asked--but I think judges for awards/contests shouldn't be comparing the stories to any story ever written. They should be comparing the stories to the other stories that are being considered in this competition, and selecting the best of *those*.

      Thanks for the thoughts!

      Delete
  2. I once judged a writing contest for short stories. I had to review each story and name all the good and bad points regarding characters, style and plot. It was quite a job, because most of the stories were pretty substandard. But I tried to be as supportive as possible and provide feedback that would help any author. That was the first round.

    There were four story categories: mystery, thriller, fantasy/SF and horror. I was one of the four judges. We had to read all the entries. Two stories were nominated in each category. Then I had to coach the two writers in the mystery category on how to write a new story to enter the competition---for the second round. Then those two stories were judged by the judges in the other three categories, and I judged the six stories in the other three categories. And all eight stories were published. People could vote for their favorite stories. The public vote and the judges' votes were added together to decide which story won the contest.

    I thought that was an interesting approach. But it was time consuming!

    ReplyDelete
    Replies
    1. Hi Anne--always good to see you here.

      I'm sure that WAS quite a job. And I admire you for it--there was much more associated with that contest than for most of those I've judged. I'm sure the writers appreciated all the coaching and knowledge they received. Thanks for the note!

      Delete
  3. I've judged everything from writing contests sponsored by regional writer groups to the Edgar and the Shamus Awards, and I think we're both reading for the same project that prompted today's post. I think each is different.

    With Edgars and Shamuses (and probably the Thrillers and a few others), all the stories/novels submitted have been professionally published, so the quality bar is automatically much higher. The goal is to select the best work from a pool of work that's already passed through gatekeepers and editors and the like. And there's "weight" behind selecting the shortlists and the winners of those Awards because those selections have the potential to impact a writer's career. An Edgar nomination (or, better still, a win) will appear on a writer's bio and/or book covers for the rest of his/her career, drawing attention from readers, editors, agents, and publishers.

    The regional writing competitions I've read for (and, yes, they had submission fees: top three received cash awards; any excess went into the organization's coffers to fund other things), the goal wasn't necessarily to select publishable work. It was to select the best of what was submitted, which may or may not have been publication ready, and encourage members to continue pursuing their dreams.

    I view the project we're working on as more like being a slush pile first reader. The goal is to select what meets the guidelines and what's publication-ready and to pass those stories onto the editor to make final selections.

    In all three cases, I have to ensure that I don't let my personal biases impact my ability to judge quality. As an editor, I might prefer to publish a certain sub-genre and may kick things out of my slush pile that don't fit within that sub-genre. As a contest or award judge, I have to judge each story on its own merits.

    It's a heck of a responsibility at any level.

    ReplyDelete
    Replies
    1. Thanks, Michael--you said all that far better than I could've. I'm especially glad you commented because you are not only a frequent judge, you're an editor. I suspect there are many, many stories you could tell us about both "jobs."

      I too have found it hard to keep personal biases out of my judging responsibilities. I'm only a writer, NOT an editor (not often, at least), but even so, I like certain genres and subgenres better than others, and every story must be judged on its merit, whether it's a cozy or a thriller or a romance.

      As for being an judge for the Edgars, I didn't say much about that competition, but you're correct, that's way different from others, and involves different ways of looking at the "entries." This whole thing is a crazy subject.

      Thanks as always.

      Delete
  4. Interesting post, John, and informative, as usual. I've judged three ss contests, and I agree that giving stories a number grade for several aspects like plot, character, etc. is time-consuming and redundant. I had to do it for my last few years teaching for a standardized test in CT and found it too restrictive. I can tell you why I think a story is good or bad (or good and better, if you prefer), and it seldom requires numbers. Some stories have a unique plot twist that's done very well, a memorable character, or effective humor. Some have terrible dialogue. So there.

    One contest asked for no number or template, but I made a spread sheet and gave each story a 1-10 and comments about why I graded it that way: weak opening; unbelievable twist; clumsy writing, whatever.

    I was one of five or six judges on that contest and the big takeaway was how much we diagreed. There were about 150 stories entered, and, since many of them may have been by novice writers, about half of them were bad to terrible. I only thought a handful were good. The winner was a story I finally put back onto my list after cutting it, and I thought it was average at best, but three other judges loved it.

    Miachel's point about the Edgars et al is a good one: stories that have been published will tend to be better because the authors have more experience and training.

    And beyond a certain point, all the arts--theater, music, fiction--become subjective anyway.

    ReplyDelete
    Replies
    1. How true, Steve. As for the Edgars, every time I've been a judge for those, the judges were surprisingly in agreement (!). That was a relief, but pretty unexpected, I think, to everyone. That has not been the case in several other instances, in other contests, but every time I think we eventually came to an agreement in the end. Yes, subjective for sure.

      I honestly cannot see the reason for this insistence on number grades and checklists, unless, as implied earlier, the authors are expecting detailed critiques. Judges should be chosen such that they can be trusted to find the best stories without the need for tons of supporting reasons. It's unnecessary, time-consuming, and maddening.

      There. Venting accomplished.

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Another thoughtful and insightful post, John! I've judged for the Edgars several times and recently was one of a panel of judges evaluating submissions for a Sisters in Crime chapter's anthology, but I don't remember ever judging anything that billed itself as a contest.

    Like you, I find the whole concept of the scoring rubric unnecessarily cumbersome, an attempt to turn a process that is by its very nature subjective into something that at least looks as if it's objective. I say humbug. You're asking me which story (or book, or script, or film) I thought was the best. I'm happy to tell you. But I don't need a bunch of artificial metrics to do that. I'm going to read (or watch) the entrants and then tell you which one I thought was best!

    Like Michael, I also edit anthologies, but I've never yet done an open-call project. He might get sixty submissions (or more) for a volume that's only going to include fifteen stories and therefore have to judge which fifteen of the sixty are best suited for his project. My anthologies are by invitation only, so for the most part the judging happens before I ever see the stories: what I'm judging is potential contributors, not actual contributions.

    Anyway, keep up the great work, Oh Sayer of the Sleuth!

    ReplyDelete
    Replies
    1. Josh, do great minds think alike or what? (Actually, since you probably taught me most of what I know, I'm not surprised we agree often.)

      Sometimes I think a lot of these competitions still use the "graded" method of selection because that's the way they've always done it. But it's still wrong.

      As for selecting stories for an anthology, which I suppose can be considered a competition, the only one I've ever edited (a mystery antho, years ago) was about half invitation and half open, and many of the open-call stories were surprisingly good, which made my job a lot easier. (As I said to Michael, I find your comments on this whole story-selection/story-evaluation subject especially valuable because you are yourself an editor.)

      Thanks as always, my friend.

      Delete
  7. I admire your patience and discipline. All that work in addition to your own writing! I don't think I could do it. I get asked to review friends' manuscripts and I find that hard enough. Plus I'm a slow reader.

    ReplyDelete
    Replies
    1. Sorry. Didn't mean for that to be anonymous. Floyd Sullivan

      Delete
    2. Hi Floyd--thanks for the comment! Though I'm not sure how much patience I have, or discipline either. What I am sure of is that I say yes to too many projects.

      As for reviewing friends' manucsripts, I feel your pain. That *is* hard--especially if they're looking for praise over encouragement. And those requests are also hard to say no to.

      I also am a slow reader. My wife isn't--she's lightning fast. Big sigh.

      Delete
    3. Floyd Sullivan30 July, 2023 13:08

      My wife, too! She reads 100 books a year, and proofreads my stuff in all her spare moment!

      Delete
    4. Look at it this way: We're smart to have married them!

      Delete
  8. I think of this blog as a MasterClass…always interesting viewpoints, advice, and tidbits. I imagine one of the hardest parts of serving as a judge would be staying quiet about it—especially if you were to ever meet one of the authors in real life. I imagine I’d have a really hard time holding myself back from telling someone how much I loved their work without giving myself away. (Someone whose story was not so great? Probably easier to keep that unspoken!) Cool to read about the different types of contests and the amount of work that goes into judging some of them.

    ReplyDelete
  9. I always forget to add my name because my phone and this website don’t get along! Above comment was Ashley-Ruth Bernier

    ReplyDelete
    Replies
    1. Ashley-Ruth, you are so kind! Glad to hear you like the blog.

      You're right, it's hard sometimes not to tell others about these judging gigs--especially for the Edgars, when the UPS man comes to your house every day with new packages for months on end.

      Seriously, though, the hardest part is always the reading, because any big contest or competition is going to have a lot of entries. And the feeling of responsibility is always there, to read each entry thoroughly enough to be sure you can give a good opinion. My fear is always that I/we might somehow overlook an entry that is deserving of the top honors--although I don't think it's ever happened.

      It's always good to hear from you. Thanks again!

      Delete
  10. I have judged quite a few deals and have been asked to do so again for an upcoming contest. Not sure how much I can say about it as I don't want to inadvertently reveal something I should not have.

    Will say that I am not a fan of the detailed scoring sheet deals because my opinion of a story does not always fit the nice and neat box system. I also have to be careful not to let my preference bias mess things up. A way around that, which may or may not work, is I tell whomever running things (my contact person) that if my answers are way outside the parameters of what they are getting from others, it might be a good idea to dump my scores.

    ReplyDelete
    Replies
    1. I'm not surprised, Kevin, that you're asked often to be a judge--I probably don't know anyone who reads more than you do.

      As for your warning to the organizers about your answers, I seriously doubt any of your scores get thrown out. I probably should issue those warnings too. I confess I've never openly complained about filling out these cumbersome and useless scoresheets--I do what I'm told--but I also believe the best competitions don't require them. I wish the others would get the word.

      Thanks for stopping in, here. Keep judging those contests!

      Delete
  11. I've been a judge for a number of awards and anthologies, and I TOTALLY agree with pretty much everything you said. That 25-50-25 split is spot on (in my experience), and I, too, don't like it when there's a complicated rubric with ten categories to evaluate each story on. What I like about judging the stories blind is the reveal surprise--when I get to see who's written the stories that have been selected.

    ReplyDelete
    Replies
    1. Good thought, Alan! The "reveal" of who wrote what is indeed interesting--and sometimes surprising, too. Yes, that is a good point, regarding the blind judging. I never thought of that.

      By the way, I'm beginning to wonder who DOES like those scoresheets. Just the organizers, probably.

      Delete
  12. I have said for years that about 20% of published stories are good enough to be nominated for an award. Among those it's just a matter of who the judges are in a given year.

    ReplyDelete
    Replies
    1. Rob, I think I remember your mentioning that percentage--and I agree.

      And yep (still on the subject of awards and nominations), who the judges are does matter.

      Delete
  13. John, I think you pretty well covered everything about judging awards and contests, plus my dislike on how some require checkbox scoring. What little you didn't cover was picked up in the comments.

    Judging short stories is easier than judging novels to find the Best Novel, especially when you receive 300+ submissions. But, I did love having all that new reading to go through. Unfortunately, for me, declining eyesight has ended my judging days.

    ReplyDelete
    Replies
    1. R.T., you're right--judging shorts is always easier. But (as you said) judging the longer fiction's fun too, because of all that (constantly) incoming reading material. One year I remember we had seven boxes of new books stacked in the corner for almost a year, and the UPS guy and I had become old friends.

      Sorry to hear about the declining eyesight. Even though judging might be out, I trust that you'll still be reading and writing for a long time to come.

      You take care, and thanks for the thoughts.

      Delete
  14. John, I've judged a lot of contests and awards, and have also made rules for myself. I will no longer fill out required checkbox scoring. After marking over 3000 manuscripts for the fiction classes I've taught, I don't feel the checkbox method is a fair way to judge fiction. It doesn't capture the 'magic' of a work, which can be all-important. I understand that the purpose of the checkbox approach was to counter racial and gender prejudice, but I think blind contests better serve that.

    ReplyDelete
    Replies
    1. I agree, Melodie. And you're correct, blind contests do a lot to address that issue--or at least to address the perception that it would be an issue. In truth, I feel that most judges would overlook the names anyway and judge the story on its value alone.

      As for the scoring systems, I judged a competition last year that was so cumbersome (because of the checkbox/spreadsheet approach) that in hindsight, I probably would've declined that friend's request to be a judge.

      Delete
  15. Oh poop! That was Melodie above.

    ReplyDelete
    Replies
    1. Leigh, make her wash her keyboard out with soap.

      Delete
    2. What can I say? poor thing...It's all the gin that's been spilled on it...

      Delete
  16. I have been an annual judge of Bulwer-Lytton Fiction Contest entries (a pre-culled final selection) by virtue of winning the 2006 Grand Prize. Sometimes my selections are not those of the other judges, but occasionally we are in perfect alignment. This year I also judged Derringer entries in Short Short Stories. Amazing how many members couldn't follow the simple entrance instructions.

    ReplyDelete
    Replies
    1. Ha! You're right, Jim. You'd think folks entering any competition would at least follow the guidelines. How hard were you on those who didn't? (I tend to be pretty unforgiving of that kind of thing, and some editors I know are as well.)

      I assume judging that pre-culled group of entries probably eases the workload a bit, right? I'm sure it's still a big task. Congratulations, by the way, on your Grand Prize!

      Delete
  17. The Derringer people who didn't follow instructions usually didn't write interesting stories. The BLFC receives thousands of entries every year. I look at 120-150 finalists spread over 12 categories.

    ReplyDelete
    Replies
    1. Whoa! Yep, quite a task.

      Delete
    2. Thanks, John. I just finished sending in my 2023 BLFC picks. If anyone would like to know the secrets of winning, they are revealed in my post to Kevin's blog on 4/03/2021.

      Delete
  18. Excellent post, John. I've judged several contests over the years, as well as the Edgars. In addition, I've founded a literary review and two mystery anthology series, which means a lot of reading. I've declined to judge any contests with forms to fill out (rating plot, character, etc.) because a good story can't be reduced to numbers in slots. Judging contests and reading for anthologies is work, but the result gives me such a boost I keep on doing it. --Susan Oleksiw

    ReplyDelete
    Replies
    1. Thank you, Susan. Yes, it is a boost, and I hope to participate in judging more competitions in the future--as I mentioned earlier, I always feel honored anytime I'm asked. But GOOD FOR YOU, for declining those gigs for contests that demand goofy story-element rating requirements. I plan to take that approach in the future.

      Delete
  19. Good for you, John! The 25-50-25 rule makes sense– it’s a bell curve!

    I’ve witnessed a couple of oddities. A regional contest had a judge fall apart perhaps suffering a mental break, apparently hating everything and everybody in every story. The competition director asked participants if they’d like to be rejudged. I chose yes, of course!

    3-4 years ago, Haboob drew my attention to a competition sponsored by Cosmopolitan Magazine, splashed in a full page ad. I believe contest rules indicated ten or a dozen honorary spots, plus 1st, 2nd, and 3rd places to be announced in 3 months or so. I entered, why not?!

    But the competition was apparently swallowed by a black hole. Nothing, nada, not a peep came after that. What the hell happened? Did no one else enter? I can’t believe that. Was no story sufficiently good to place? I find that hard to believe. Did the contest violate some kind of law? Something messy enough they couldn’t resolve it? Enquiring minds want to know.

    Good luck in all that reading, John. I agree that 5 pages or fewer is sufficient to realize what a reader is getting into.

    ReplyDelete
    Replies
    1. Leigh, sounds like you effectively killed that contest, by entering!! Seriously, you have to wonder if Cosmo just changed their minds about the whole deal, and the judging never happened. A mystery indeed.

      I'm really surprised by that "oddity" you mentioned. How would you like to be that contest director, having to decide what to do to rescue the situation?

      As for me and my upcoming gig, thank you for the well wishes--I'm actually looking forward to it. All indications are that it'll be a well-run competition, and I'm hoping we'll have some fantastic stories to judge.

      As always, thanks so much for your thoughts!

      Delete
  20. John, I'm with you on checking boxes, and when asked to do so—as in recent years when judging the Derringers—I confess I take it with a grain of salt. The stories I think are the best come out the highest on the bottom line. Judging the Edgars has taught me a lot. The contenders tend to leap out, even in a field of 600 novels, because they stand firmly on all three legs of what I think of as the three-legged stool: writing, character, and plot or structure. As for reading or submission fees, I've just had an interesting experience. I've been writing poetry as well as short stories this summer (once very much my bag as a writer, but I'm surprised to find myself doing it again). I was just asked to blurb a poetry book by an old friend, a multiple award-winning poet with numerous books to his credit. When I took a look at his publisher's submission guidelines, I saw they charge a reading fee. So I asked my friend about it. He said he finds that route "more reliable" than contests. I said, "So even a very well established poet doesn't eschew reading fees?" He said, "Not any more." Things are tough all over. Of course, in the poetry world, "they're supposed to pay you" is and always was an absurdity, so it's a little different from the fiction world.

    ReplyDelete
    Replies
    1. Interesting, Liz. I know very little about the poetry world, even though I've sold several hundred to magazines over the years and had a collection of 300 poems published not long ago. (My poems are just light-verse examples of fun and wordplay.) It's surprising, though, to hear of your old friend's views and experiences. I can't see myself ever paying a submission fee, for anything,

      And you're right, judging for the Edgars DOES teach you a lot.

      Thanks so much for sharing all this!

      Delete
  21. Ha! Probably not, Jeff, because I'm reluctant to say no to some of these requests.

    ReplyDelete

Welcome. Please feel free to comment.

Our corporate secretary is notoriously lax when it comes to comments trapped in the spam folder. It may take Velma a few days to notice, usually after digging in a bottom drawer for a packet of seamed hose, a .38, her flask, or a cigarette.

She’s also sarcastically flip-lipped, but where else can a P.I. find a gal who can wield a candlestick phone, a typewriter, and a gat all at the same time? So bear with us, we value your comment. Once she finishes her Fatima Long Gold.

You can format HTML codes of <b>bold</b>, <i>italics</i>, and links: <a href="https://about.me/SleuthSayers">SleuthSayers</a>