ADR-010: Spider Normalized Requirement Scoring
Date: 2026-05-28
Status: Accepted
Deciders: Engineering
Affected areas: Spider scoring engine, lib/api-spec, artifacts/api-server, artifacts/ftm-web
Context
Spider's skill-match axis previously read project_slots.requiredSkills text[] — a free-text array with no referential integrity, no proficiency metadata, and no criticality weighting. Phase 1A-schema introduced slot_skill_requirements with typed, catalogued skill references, proficiency floors, criticality levels, and a verification-required flag. Phase 1D wires those normalized requirements into a parallel scoring path and exposes the diagnostic signals through the API and UI.
Decision 1 — Preferred input: slot_skill_requirements over free-text array
When slot_skill_requirements rows exist for a slot, the normalized scorer reads from that table instead of project_slots.requiredSkills. This provides typed skill IDs, criticality weights, proficiency floors, and verification flags.
Rationale: Free-text matching is lossy (case mismatch, aliases, abbreviations). Normalised references enable deterministic, auditable matching and allow skill taxonomy relationships to contribute partial credit.
Decision 2 — Legacy fallback when no normalised requirements exist
If no slot_skill_requirements rows exist for a slot, the scorer falls back to the legacy free-text path. The API response omits requirementSource, scoringMode, and all explanation arrays in this case — the consumer sees exactly the same shape as before, preserving full backward compatibility.
Rationale: Slots are normalised asynchronously. Forcing normalized mode on un-migrated slots would produce incorrect zero scores during the transition.
Decision 3 — Criticality weights: required > preferred > nice_to_have
| Criticality | Weight |
|---|---|
required | 1.00 |
preferred | 0.60 |
nice_to_have | 0.30 |
A missing required skill caps the slot's skill-match contribution more aggressively than a missing preferred skill. Weights are constants (not per-tenant flags) in Phase 1D; per-tenant tuning is deferred.
Rationale: Fixed weights allow auditable, reproducible scoring. The values reflect common recruiting practice where required skills are binary gatekeepers.
Decision 4 — Proficiency depth: scaled penalty, not hard cutoff
When a candidate's proficiency for a required skill is below the slot's minimumProficiency, a graded penalty is applied proportional to the number of levels below the requirement (gapLevels). A one-level gap reduces the skill's contribution by ~25%; a two-level gap by ~50%; three or more levels by ~75%.
Hard cutoffs (pass/fail on proficiency) were rejected because they create cliff-edge scoring that is hard to explain to hiring managers and produces wide score swings on rounding.
Rationale: Continuous penalties are more stable and explainable. The proficiencyGaps[] array surfaces the exact penalty so reviewers can audit decisions.
Decision 5 — Verification signal: additive penalty, not disqualifier
When verificationRequired = true and the candidate has no verified evidence for the skill (no endorsement, certification, or assessment record), a penalty of −10% to −20% is applied to that skill's contribution. The penalty magnitude is fixed in Phase 1D; per-slot configuration is deferred.
Verification failure does not disqualify a candidate. Recruiters remain in control of the final decision.
Rationale: Verification data is sparse in early platform usage. Hard disqualification would suppress all candidates on newly-created slots, harming hiring manager experience.
Decision 6 — Cert-inferred hierarchy: CertificationSkillMappings drive partial credit
A candidate holding a certification whose certification_skill_mappings record maps it to a required skill receives partial credit. The credit is mapping.confidence * 0.8 (capped at 0.8) for that skill's contribution. The inferredCertificationMatches[] array records which cert granted which partial credit.
Rationale: Certifications are durable, verifiable proxies for skill possession. Ignoring them penalises candidates who hold formal credentials. The 0.8 cap ensures direct skill evidence always scores higher than inferred evidence.
Decision 7 — Relationship partial credit: taxonomy graph edges contribute reduced score
When a candidate possesses a skill that is a parent, child, related, or equivalent skill of a required skill (via skill_relationships), partial credit is applied based on relationship type:
| Relationship type | Partial credit |
|---|---|
equivalent_to | 0.90 |
parent_of | 0.70 |
child_of | 0.60 |
related_to | 0.40 |
prerequisite_of | 0.50 |
part_of | 0.55 |
Only the highest-credit relationship is used per required skill (no double-counting).
Rationale: Skill taxonomies capture real substitutability. Ignoring them produces artificially low scores for candidates with deep expertise in adjacent technologies.
Decision 8 — Feature flag modes: legacy | shadow | normalized
The scoring mode is controlled by a feature flag (spider_scoring_mode) per tenant (or globally). Three values are supported:
legacy(default post-merge): Original scorer only. No normalized code paths run. API fields absent.shadow: Both scorers run. Legacy score is returned in the API response. Normalized score and comparison delta are written toaudit_logswithaction = 'spider_shadow_comparison'. No UI impact.normalized: Normalized scorer wins. Legacy scorer still runs and its result is stored for comparison.
Rationale: Gradual rollout with a delta-review gate prevents surprise score changes reaching hiring managers. The three-stage progression (legacy → shadow → normalized) mirrors standard feature-flag practice for scoring changes.
Decision 9 — Shadow behaviour: audit log, not DB score column
Shadow comparison results are stored in audit_logs.after as a JSON payload (not in a separate table or on match_scores). The GET /api/admin/catalogue/spider-shadow-comparisons endpoint aggregates them.
Rationale: A separate table would require a migration and ongoing maintenance. audit_logs already has the right lifecycle (write-once, queryable, no FK constraints). The admin endpoint provides sufficient aggregation for the delta-review gate.
Decision 10 — Normalization mode: normalized scorer result replaces legacy in API
In normalized mode the match_scores row is updated with the normalized scorer's output. The API response includes requirementSource: "normalized" and scoringMode: "normalized". Legacy score is still computed and stored in audit_logs for rollback diagnostics.
Rationale: Consumers (company UI, submission reports) read from match_scores. Writing both scores to separate columns would double the schema surface area and complicate client code. A single authoritative score with provenance fields is simpler.
Decision 11 — No legacy removal
project_slots.requiredSkills text[] is preserved and continues to be the input for legacy mode. No migration removes or empties this column. The column is the fallback if normalization is incomplete.
Rationale: Removing legacy data before normalized coverage is 100% would break scoring for un-migrated slots.
Decision 12 — Cert enum stored as text
certifications.name is stored as text, not a platform-managed enum. The cert-inferred hierarchy relies on certification_skill_mappings records, not on enum-based dispatch. New certifications are added by inserting rows, not by migrating enum values.
Rationale: Postgres enum migration requires DDL changes; text allows the catalogue to grow without schema changes. Validation is enforced at the application layer via reviewStatus.
Decision 13 — Relationship taxonomy staleness self-heals on 60-minute sweep
When a skill relationship is modified (new row, deleted row, weight change), affected match_scores rows are marked isStale = true. A background sweep re-scores stale matches on a 60-minute cadence. There is no immediate re-score on taxonomy changes.
Rationale: Immediate re-scoring on every taxonomy edit would saturate the queue during bulk catalogue updates. The 60-minute window is acceptable given that taxonomy changes are infrequent and match scores are advisory rather than binding.
Known Limitations
- Cert enum as text — no referential integrity between cert name strings across tables; resolves when full catalogue is normalised.
- Relationship taxonomy staleness self-heals on 60-min sweep — scores may be temporarily stale after a taxonomy edit.
- No per-axis flag granularity —
spider_scoring_modeis a single flag covering all axes. Per-axis flags (e.g., normalized skill-match + legacy culture-fit) are deferred to a future phase. - Verification data is sparse — the verification gap signal may have low coverage until the verification workflow (Phase 2) is complete.
- Partial credit caps are fixed — per-slot or per-tenant tuning of relationship and cert credit multipliers is out of scope for Phase 1D.