Judging workflows

Judging is chapter II of the rail and holds three modules: Judging Panel, Scoring and Conflict Checker. This guide covers setting the scoring rules, adding and assigning judges, getting them into the portal, chasing the ones who have not finished, and what the conflict tooling can and cannot see.

Before you start

• Your categories exist, and each one has a Scoring Method and any weighted criteria set on the category record.
• You have nominations that have been through the Review Queue. Judges score what has cleared review, so an empty panel usually means an empty queue.
• You have each judge's name, email and the categories they should see.

The tabs on Judging Panel

Nine tabs, and you will use the first three most: All Judges, Scoring Progress, Category Matrix, Score Analysis, Deliberation, COI Declarations, Settings, Embeddable Widget and Scoring Portal.

1. Set the scoring rules first

Open the Settings tab before you invite anybody, because changing the scale after judges have started makes their existing scores hard to compare.

• Scoring Method — 1-10 Scale, 1-5 Scale, 1-100 Scale, Ranked Order or Pass/Fail.
• Score Weighting — Equal Weight, Category Chair x2, or Custom Weights.
• Minimum Judges per Category and Maximum Categories per Judge — the two numbers that stop a category being decided by one person and stop one person being asked to read everything.
• Scoring Deadline and Deliberation Deadline.
• Blind Review (hide nominee names) — this is the anonymity control. If your process requires judges not to know who they are scoring, this is the switch that does it.
• Allow Judge Comments, Require Justification for Extreme Scores and Enable Score Calibration Alerts — the three that make a decision defensible afterwards.

Per-criterion weighting is not set here. It is set on each category, as Criteria Name and Weight rows — see Categories and nominations.

2. Add the judges

Judging Panel → Add Judge. The form takes Judge Photo, Full Name, Email, Organisation, Title / Role, Phone, Expertise Area, Assign to Categories, Bio / Notes, and two checkboxes: Send invitation email immediately and Require COI declaration. Status is Active, Pending, Suspended or Removed.

Assign categories at this point rather than later. The All Judges table shows Name, Organisation, Title, Email, Categories, Scores, Completion and Status, and a judge with no categories will sit at 0% completion forever without an obvious reason why.

3. Configure the Scoring Portal

The Scoring Portal tab holds the Portal URL your judges use and its settings: Authentication (Magic Link (Email), Password or SSO), Portal Branding (Event Branding, Minimal / White Label or Custom) and Session Timeout (30 Minutes, 2 Hours, 8 Hours or No Timeout).

Magic link is the default. If your judges are scoring long entries in one sitting, move the session timeout up — 30 minutes is the most common cause of "it logged me out and I lost my scores".

4. Set the chasing up before you need it

Also on the Scoring Portal tab: Auto-Remind Frequency (Daily, Every 3 Days, Weekly, Disabled), Reminder Start (Days Before Deadline), Completion Notification (Email Admin on 100%, Email Admin on Each Score, Disabled) and Outlier Alert (flag scores 2+ or 3+ standard deviations from the mean, or disabled).

5. Watch it land

Scoring Progress shows completion per judge. Category Matrix shows it the other way round, per category, which is how you spot a category sitting below your Minimum Judges per Category with a week to go. Score Analysis is where outlier flags surface, and Deliberation is for the panel conversation before the slate is set. From Winners Slate you can also use Chase Remaining Judges and View Judge Scores.

Conflicts of interest: what the checker actually sees

Tick Require COI declaration on a judge and their declaration appears under the COI Declarations tab. The separate Conflict Checker module scans on demand — press Run Full Scan or Quick Scan (Active Only); nothing runs on a schedule.

Be clear about its limits, because they matter. It can see a judge and a nominee sharing a surname, a judge scoring in a category where their own organisation has an entry, and a sponsor appearing on both sides of a category. It has no connection to LinkedIn and none to Companies House, so a conflict arising from a directorship, an old employer or a personal relationship will not be found by this screen. It also does not remove anything from a total on its own — a flagged conflict is a prompt for you to act, not an automatic exclusion. Withdraw the judge from the category, or record the resolution, and do it yourself.

Common problems

• A judge sees no entries. They have no categories assigned, or nothing in those categories has cleared the Review Queue.
• Completion sticks below 100%. Check Maximum Categories per Judge — a judge assigned more categories than the cap can be unable to reach the rest.
• Scores look inconsistent between panels. Turn on Score Calibration Alerts and check Score Analysis before the slate, not after.
• You changed the scale mid-way. There is no conversion. Decide the scale in step one.