Should we replace our manual QA inspection line with machine vision? 200K units/month, 3 shifts, 0.2% defect rate, $400K budget for automation.
Assumptions used in this verdict
The Council identified missing context and filled in sensible defaults. The verdict above is based on these assumptions:
Pro users can specify these constraints and get a tailored verdict instead of defaults.
Optional considerations
The Council noted 2 optional considerations that were not needed to reach the final verdict. These are advisory only and do not affect the completed outcome.
- By what date do you need the machine-vision line making pass/fail decisions on the 200K units per month, and is that date driven by a customer commitment, a labor-change event, or a quality escape you cannot absorb much longer?
- How many internal engineers can you actually assign to build, validate, and then support a vision system across 3 shifts—specifically controls, software/vision, manufacturing engineering, and maintenance—and how much of that would still be available after launch?
This verdict assumes 64% of constraints
Constraint slots are tagged by provenance so synthetic defaults do not look like observed facts:
- synthetic timeline synthetic default (not observed): no hard deadline assumed (not_addressed)
- user_provided budget provided: provided by user (user_provided)
- synthetic reversibility synthetic default (not observed): assumed reversible within 90 days (not_addressed)
- synthetic team size synthetic default (not observed): standard team (5-10 engineers) (not_addressed)
- user_provided current scale provided: provided by user (user_provided)
- synthetic existing stack synthetic default (not observed): greenfield assumed (not_addressed)
- synthetic domain operating environment synthetic default (not observed): operating environment not specified (not_addressed)
- user_provided domain failure tolerance provided: provided by user (user_provided)
- user_provided domain latency or cycle time provided: provided by user (user_provided)
- user_provided domain measurement basis provided: provided by user (user_provided)
- synthetic domain actuator or process constraints synthetic default (not observed): actuator/process constraints not specified (not_addressed)
- synthetic domain degraded mode synthetic default (not observed): degraded or fail-safe mode not specified (not_addressed)
- synthetic domain human override synthetic default (not observed): human override boundary not specified (not_addressed)
- synthetic domain validation evidence synthetic default (not observed): validation evidence not specified (not_addressed)
Analysis-locked
This is not a final verdict. The Clerk capped this filing to risk inventory and evidence plan; recommendations require a more grounded filing.
Do NOT fully replace the manual QA line
Systematic missing-items audit
Decision
Critical unknown cap
Confidence capped at 39% because Manufacturing design is missing required grounding facts: operating_environment, actuator_or_process_constraints, degraded_mode, human_override, validation_evidence.
- operating environment: What operating environment does this run in? Include facility, field conditions, jurisdiction, or deployment context.
- actuator or process constraints: What actuator, equipment, capacity, material, route, or enforcement constraints bound the decision?
- degraded mode: What degraded, fallback, exception, or fail-safe mode exists when the system cannot continue normally?
- human override: Who can override or stop the system, and under what condition?
- validation evidence: What validation evidence exists? Include simulation, pilot data, field tests, acceptance tests, or historical outcomes.
Candidate estimate (inferred, not source-confirmed): Do NOT fully replace the manual QA line. Deploy machine vision in parallel shadow mode for 9 months, then graduate individual defect classes to vision-authority only when they individually clear 300+ captured instances with <0.5% false-reject and zero escapes.. Because at 0.2% defect rate you generate only ~400 defects/month total, which is structurally insufficient to validate a vision system's catch rate across multiple defect classes — the rule-of-three requires ~300 captured defects with zero misses to claim 99% detection at 95% confidence, creating a minimum 9-month observation window before any class can be trusted to vision-only authority.. Key failure modes: Supplier lot change shifts surface reflectivity causing false-positive surge that overwhelms exception queue; Rare functional defect classes never accumulate statistical validity and create permanent hybrid complexity; Total captured defects <250/month even at full volume means vision authority is unprovable on ANY class — project caps at advisory-only; Per-class sample starvation when defects span 8+ classes drops individual class counts below validation floor. Thresholds: ~400 defects/month at 200K units × 0.2%, 300 captured defects with zero misses per class for 99% catch at 95% confidence, 9-month minimum observation window, <0.5% false-reject gate (=~1000 scrapped good units/month), $180K capex + $130K/yr vision-engineering retainer + $90K reserve, <250 total monthly captured defects = hard kill condition
Inferred specifics
| Value | Kind | Basis | Where introduced |
|---|---|---|---|
| at 0.2 | version | synthetic | chosen_path |
| mode for 9 months | estimate | synthetic | chosen_path |
| individually clear 300+ captured instances with <0 | estimate | synthetic | chosen_path |
| instances with <0.5% false-reject and zero escapes | threshold | synthetic | chosen_path |
| generate only ~400 defects/month total | estimate | synthetic | chosen_path |
| rule-of-three requires ~300 captured defects with zero | estimate | synthetic | chosen_path |
| to claim 99% detection at 95% confidence | threshold | synthetic | chosen_path |
| detection at 95% confidence | threshold | synthetic | chosen_path |
| captured defects <250/month even at full volume | estimate | synthetic | chosen_path |
| defects span 8+ classes drops individual class | estimate | synthetic | chosen_path |
| =~1000 scrapped good units/month | config | synthetic | chosen_path |
| $180K capex + $130K/yr vision-engineering | estimate | synthetic | chosen_path |
| capex + $130K/yr vision-engineering retainer + $90K | estimate | synthetic | chosen_path |
| retainer + $90K reserve | estimate | synthetic | chosen_path |
| the last 12 months | estimate | synthetic | next_action |
| only 0.42 | version | synthetic | selection_rationale |
| 0.93 | estimate | synthetic | selection_rationale |
| across rounds 2-4 by all three models | estimate | synthetic | selection_rationale |
| b004 had the highest confidence | estimate | synthetic | selection_rationale |
| 400 defects/month is insufficient for multi-class validation | estimate | synthetic | selection_rationale |
Highest-probability failure mode (within analysis ceiling)
What happens: After shadow mode deployment, a supplier lot change shifts material surface properties (reflectivity, color, texture) causing the vision model's baseline reference to drift — units previously classified as normal now trigger anomaly flags while new lot's normal variation masks a genuine emerging defect class.
Why hidden: Shadow mode success metrics are computed against the CURRENT supplier lot and process state. A lot change is a discrete event that invalidates the confusion matrix built over months, but the system reports no degradation until the new lot's data flows through — by which time the manual line has already dispositioned the units and the vision system's silent disagreement is only visible in retrospective log analysis.
Detection gap: Standard monitoring tracks aggregate false-reject and escape rates averaged over days/weeks. A sudden lot-change-induced distribution shift may not trigger alerts if (a) the shift is gradual within a lot or (b) alert thresholds are set on weekly rolling averages rather than per-lot boundaries.
Blast radius: If the class has already graduated to vision-authority, escaped defects from the new lot ship before the per-lot confusion matrix is rebuilt. At 200K units/month, even 1-2 days of undetected escape at a new-lot defect rate can release 100+ defective units.
Monitoring signal to add: Track vision system confidence-score distribution per incoming material lot ID. Alert when median confidence drops >1 standard deviation from the prior lot's baseline within the first 500 units of a new lot. Cross-reference with incoming material cert changes.
Operational feasibility
Next actions
Verdict-to-Work
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
Export as markdown
What usually goes wrong
- Winner appears obvious early — hidden variables emerge later
- Benchmark conditions don't match production reality
- Optimization target shifts after commitment
- Risk assessment focused on known threats, missed novel vectors
- Compliance checkbox passed but operational security remained weak
- Low-probability high-impact scenario treated as negligible
- Hidden constraints surface late and invalidate the sacrifice
- One variable dominates unexpectedly under load
- Regret when assumptions shift — the 'wrong' option becomes viable
Council notes
Structural integrity
All required personas contributed to all required rounds.
Contribution matrix
| Role | seed | round 1 | round 2 | round 3 | round 4 |
|---|---|---|---|---|---|
| Architect | present | present | present | present | present |
| Engineer | present | present | present | present | present |
| Analyst | present | present | present | present | present |
Injections
Attack grid ⓘSurvival rate shows how the recommendation holds under stress scenarios. Low scores indicate conditional vulnerability, not a flaw in the recommendation.
Evidence boundary
Observed from your filing
- Should we replace our manual QA inspection line with machine vision? 200K units/month, 3 shifts, 0.2% defect rate, $400K budget for automation.
Assumptions used for analysis
- Candidate estimate (inferred, not source-confirmed): 0.2% defect rate is accurate and represents true escapes, not under-detection by current manual process
- Candidate estimate (inferred, not source-confirmed): Defects span multiple classes rather than concentrating in 1-2 easily characterized types
- Candidate estimate (inferred, not source-confirmed): $400K budget is primarily one-time capex with limited tolerance for recurring operational costs
- Current manual QA line is the ground-truth authority (i.e., manual inspectors are catching what exists)
- Production environment has supplier lot changes and process variation that would challenge a static vision model
- timeline synthetic default (not observed): no hard deadline assumed [synthetic] (not_addressed)
- reversibility synthetic default (not observed): assumed reversible within 90 days [synthetic] (not_addressed)
- team size synthetic default (not observed): standard team (5-10 engineers) [synthetic] (not_addressed)
- existing stack synthetic default (not observed): greenfield assumed [synthetic] (not_addressed)
- domain operating environment synthetic default (not observed): operating environment not specified [synthetic] (not_addressed)
- domain actuator or process constraints synthetic default (not observed): actuator/process constraints not specified [synthetic] (not_addressed)
- domain degraded mode synthetic default (not observed): degraded or fail-safe mode not specified [synthetic] (not_addressed)
- domain human override synthetic default (not observed): human override boundary not specified [synthetic] (not_addressed)
- domain validation evidence synthetic default (not observed): validation evidence not specified [synthetic] (not_addressed)
Inferred candidate specifics
- Candidate estimate (inferred, not source-confirmed): Do NOT fully replace the manual QA line. Deploy machine vision in parallel shadow mode for 9 months, then graduate individual defect classes to vision-authority only when they individually clear 300+ captured instances with <0.5% false-reject and zero escapes.. Because at 0.2% defect rate you generate only ~400 defects/month total, which is structurally insufficient to validate a vision system's catch rate across multiple defect classes — the rule-of-three requires ~300 captured defects with zero misses to claim 99% detection at 95% confidence, creating a minimum 9-month observation window before any class can be trusted to vision-only authority.. Key failure modes: Supplier lot change shifts surface reflectivity causing false-positive surge that overwhelms exception queue; Rare functional defect classes never accumulate statistical validity and create permanent hybrid complexity; Total captured defects <250/month even at full volume means vision authority is unprovable on ANY class — project caps at advisory-only; Per-class sample starvation when defects span 8+ classes drops individual class counts below validation floor. Thresholds: ~400 defects/month at 200K units × 0.2%, 300 captured defects with zero misses per class for 99% catch at 95% confidence, 9-month minimum observation window, <0.5% false-reject gate (=~1000 scrapped good units/month), $180K capex + $130K/yr vision-engineering retainer + $90K reserve, <250 total monthly captured defects = hard kill condition
- Candidate estimate (inferred, not source-confirmed): Produce a defect taxonomy document listing every defect class with its monthly occurrence count over the last 12 months, categorized as dimensional/cosmetic/functional — this single artifact resolves the three-outcome decision tree and determines whether any class can reach statistical validation within budget.
- b004 had the highest confidence (0.93) among survivors and was consistently strengthened across rounds 2-4 by all three models. While prosecution noted it 'hides a NO inside a methodology narrative' and its defense quality was only 0.42, the branch's core statistical argument (400 defects/month is insufficient for multi-class validation) is mathematically sound and directly answers the binary question: full replacement NO, partial replacement conditionally YES after per-class validation. The killed b009 (0.86) was a strong salvage candidate with an excellent three-outcome decision tree — its specifics (per-class frequency as the single blocking fact, three-outcome structure) are incorporated into chosen_path. b002 (0.65) was substantively similar but less rigorous in its statistical grounding.
- Candidate estimate (inferred, not source-confirmed): Full replacement of manual QA with machine vision within $400K budget
- Candidate estimate (inferred, not source-confirmed): At 400 defects/month, there is no way to validate detection performance with statistical confidence before committing quality authority to the vision system. Full replacement creates undetectable quality escapes that surface only after supplier or process changes.
- Candidate estimate (inferred, not source-confirmed): Build parallel QA telemetry platform first, defer all vision deployment (b007)
- Candidate estimate (inferred, not source-confirmed): Adds unnecessary delay layer — telemetry collection can happen simultaneously with shadow-mode vision deployment rather than sequentially. Defense quality was lowest (0.58) and the branch offered no specific timeline, ROI benchmark, or decision gate.
- Candidate estimate (inferred, not source-confirmed): Deploy anomaly-detection-based vision (Cognex ViDi) instead of class-specific models (b006 salvage)
Unknowns blocking a firmer verdict
- Number of distinct defect classes and their individual monthly frequencies
- Current manual inspection false-reject rate (to determine opportunity for improvement)
- Labor costs in USD per month for the 3-shift manual QA team
- Cheapest validation: Pilot a single-defect-class vision system on a Cognex In-Sight 2800 or Keyence CV-X platform for 1 SKU at full production speed and capture at least 300 defect instances over 8 weeks to verify detection and false-reject rates.
- Question assessment: This question is answerable as asked, but would benefit from refocusing to: 'Can machine vision be statistically validated to fully replace manual QA for any defect class under $400K automation budget?'
- Candidate estimate (inferred, not source-confirmed): Number of distinct defect classes and per-class monthly frequency — this single fact determines whether partial replacement is viable in 6-9 months or statistically impossible
- Whether defects are dimensional, cosmetic, or functional (functional defects may require test equipment beyond machine vision)
- Current manual inspection false-reject baseline — needed to benchmark vision system improvement
- Number of SKUs and changeover frequency — high-mix environments degrade vision system performance
- Candidate estimate (inferred, not source-confirmed): Actual labor cost of 3-shift manual QA line — needed for ROI calculation to justify $130K/yr recurring retainer
- Candidate estimate (inferred, not source-confirmed): Verdict is primarily model-reasoning; rule-of-three threshold (300 samples) is a standard statistical method but the specific application to machine vision validation gates is not externally grounded in the evidence provided
- Candidate estimate (inferred, not source-confirmed): Whether the $400K budget is one-time capex only or can accommodate recurring operational costs — if capex-only, the $130K/yr retainer creates a budget gap
- Team capacity for vision engineering and systems integration not declared in filing
- Candidate estimate (inferred, not source-confirmed): operational feasibility gap remains unresolved for Install and configure vision cell (Cognex In-Sight 2800 or Keyence CV-X) with lighting, fixturing, and integration to conveyor
- synthetic default is not observed evidence: timeline defaulted to no hard deadline assumed
- Candidate estimate (inferred, not source-confirmed): synthetic default is not observed evidence: reversibility defaulted to assumed reversible within 90 days
- Candidate estimate (inferred, not source-confirmed): synthetic default is not observed evidence: team_size defaulted to standard team (5-10 engineers)
- synthetic default is not observed evidence: existing_stack defaulted to greenfield assumed
- synthetic default is not observed evidence: domain_operating_environment defaulted to operating environment not specified
- synthetic default is not observed evidence: domain_actuator_or_process_constraints defaulted to actuator/process constraints not specified
- synthetic default is not observed evidence: domain_degraded_mode defaulted to degraded or fail-safe mode not specified
- synthetic default is not observed evidence: domain_human_override defaulted to human override boundary not specified
- synthetic default is not observed evidence: domain_validation_evidence defaulted to validation evidence not specified
- The Council could not reach a defensible verdict. Block reasons: confidence below threshold
Coverage report
| Axis | Status | Examples from verdict | Gaps |
|---|---|---|---|
| correctness | addressed | Rule-of-three: 3/n bound requires ~300 captured defects with zero misses At 400 defects/month spanning multiple classes, per-class samples may fall below 50 |
|
| operability | addressed | $180K capex + $130K/yr retainer + $90K reserve fits initial budget but creates ongoing cost Manual line retained as authority eliminates throughput risk during validation |
|
| failure_modes | addressed | Supplier lot change shifts reflectivity → false positive surge Classes <20/month never reach 300-sample floor 0.5% false-reject = ~1000 scrapped good units/month |
Fragility signals
- Hubris: RE_PASS
Operational signals to watch
Branch battle map
Battle timeline (4 rounds)
Minority report
Reject the binary. Deploy a HYBRID staged inspection, not full replacement. Phase 1 (months 0-6, ~$150K): install machine vision (Cognex In-Sight 2800 or Keyence CV-X) ONLY on the dimensional/cosmetic defect classes that are high-volume, repeatable, and have clear visual signatures — let the system auto-pass the obvious-good and auto-fail the obvious-bad. Route the ~2-5% uncertain/borderline units to a single retained manual inspector per shift (cuts headcount but keeps human judgment on the long-tail). This is mandatory because at a 0.2% defect rate you only generate ~400 defects/month, which is structurally insufficient to train and validate a vision model on rare defect classes — full replacement creates a quality escape that surfaces only after tool wear or supplier lot change. Hard gate before any Phase 2 expansion: the vision system must demonstrate <0.5% false-reject AND >99% true-defect-catch rate measured on a HIGH-MIX day across a full supplier-lot change and a shift change, not a single-SKU demo. If false-reject exceeds 0.5% (=1000 scrapped good units/month), the recovery economics invert and you stop. Budget reserves $100K for a recurring vision-engineering retainer (~$130K/yr fully loaded) — capex-only budgeting is the hidden trap. BLOCKING MISSING FACTS that must be supplied before committing: (1) number of distinct SKUs and changeover frequency, (2) number of defect classes and per-class monthly frequency, (3) whether defects are dimensional, cosmetic, or functional, (4) current manual false-reject baseline to benchmark against.
Pre-mortem skipped
Censor oversight
BLOCKED
The winning branch (b004) provides a detailed analysis but effectively dodges the original question by reframing it around statistical detectability rather than directly answering whether to replace the manual QA inspection line with machine vision. The answer also introduces a new condition (deploying in parallel shadow mode) that wasn't directly asked. While it offers valuable insights, it doesn't directly address the question's expected decision format. Other surviving branches (b002, b003, b007) also either reframe the question or suggest hybrid approaches rather than directly answering the binary choice posed. question fit score 0.45 is below semantic threshold for binary_choice/decision; verdict answered the wrong target
Structural issues
- SCOPE DODGE
- CONSULTING FOG
- Question-fit guard blocked finalization because the verdict did not answer the filed question
Evidence source proof
Evidence sources: not shown on free tier - upgrade to see which sources contributed.
Reproducibility manifest
{
"schema_version": "vectorcourt.provenance.v1",
"verdict_id": "6d653f8d-36e4-4991-8bda-eb822c6bd094",
"case_id": "6d653f8d-36e4-4991-8bda-eb822c6bd094",
"generated_at": "2026-06-27T03:21:52Z",
"protocol_version": "1.0.0",
"prompt_hashes": {
"filing_hash": "sha256:8eab5ee772f53830cce6f0736046789ba7d149c1e4ccd83954ab752832d8f208",
"council_seed_hash": "sha256:9e8e71127a0507e4a887c2c018b6b0e097ee1176c5a2731c793ac787dcfb5422",
"prosecutor_hash": "sha256:7a62169e3df54fcc52bedf9837612fb6db889a68269e412c4e15e771b570729a",
"compressor_hash": "sha256:a6cd2f86de7a4ba6e08efcd0f717c650b9bd31174bbf4ee8e903df35b3704ad8",
"nemesis_hash": "sha256:c1443b8fbd16e898b0efc38f903eeea388a77cc0829c01b72af97e6a98e14e2d"
},
"public_key_url": "https://vectorcourt.com/public-key"
}