Test plan and proposal

Why this matters

A good project proposal does not only say what will be built. It says how success will be proven and what could go wrong. Examiners will ask how a requirement will be tested and what happens if the camera fails in flight. A team that can answer from its own documents is ready to start building next semester.

The V model

Each layer on the left has a matching verification on the right. System requirements are verified by system tests, and user needs are confirmed by validation. The actual test sequence for a drone follows the ladder from SITL and HITL to field testing covered in UAT 322.

Requirements verification matrix (RVM)

The NASA Systems Engineering Handbook records verification in a requirements verification matrix that lists each requirement, its method, the test activity, the pass criterion and the result. The matrix is checked in two directions: every requirement must have a verification activity, and every activity must verify some requirement.

requirements = ["R1", "R2", "R3", "R4", "R5a", "R5b", "R6", "R8"]
rvm = [  # (activity, requirements verified, method)
    ("T1 labelled image set", ["R1"], "Test"), ("T2 timed SITL round", ["R4"], "Test"),
    ("T3 GCP survey", ["R6"], "Test"), ("A1 storage estimate", ["R5a"], "Analysis"),
    ("D1 report walkthrough", ["R5b", "R8"], "Demonstration"), ("T4 motor noise log", [], "Test"),
]
covered = {r for _, reqs, _ in rvm for r in reqs}
missing = [r for r in requirements if r not in covered]
orphans = [name for name, reqs, _ in rvm if not reqs]
print(f"coverage {len(covered)}/{len(requirements)} = {len(covered) / len(requirements):.0%}")
print("requirements without verification:", missing)
print("activities with no requirement:", orphans)

coverage 6/8 = 75%
requirements without verification: ['R2', 'R3']
activities with no requirement: ['T4 motor noise log']

Risk and FMEA

FMEA (failure modes and effects analysis), as defined in IEC 60812:2018, works through how each part can fail, what the effect is and how the failure can be prevented or detected. The traditional method rates severity (S), occurrence (O) and detection (D) from 1 to 10 and multiplies them into an RPN. The AIAG-VDA FMEA Handbook (2019) replaced this with Action Priority (AP), because equal RPNs can come from very different severities.

fmea = [  # (failure mode, S, O, D)
    ("GNSS loss near tall trees", 8, 4, 3),
    ("companion computer hangs", 6, 3, 4),
    ("battery sag in wind", 9, 2, 2),
    ("AI misses a crack", 5, 5, 6),
    ("camera trigger drift", 4, 5, 5),
]
ranked = sorted(fmea, key=lambda row: row[1] * row[2] * row[3], reverse=True)
for mode, s, o, d in ranked:
    flag = "  <- severity 9+: act regardless of RPN" if s >= 9 else ""
    print(f"RPN {s * o * d:>3}  S{s} O{o} D{d}  {mode}{flag}")

RPN 150  S5 O5 D6  AI misses a crack
RPN 100  S4 O5 D5  camera trigger drift
RPN  96  S8 O4 D3  GNSS loss near tall trees
RPN  72  S6 O3 D4  companion computer hangs
RPN  36  S9 O2 D2  battery sag in wind  <- severity 9+: act regardless of RPN

The project proposal

The proposal brings together the deliverables of modules 1–5. The schedule, budget and project risks use the tools from UAT 316 (WBS, critical path and risk register). The key principle is that every part must trace back to the requirements, and the proposal must state the conditions under which the project will be stopped or reviewed.

Module lab

Common mistakes

Summary

• The V model pairs every design layer with verification at the same level
• An RVM must cover every requirement, and every activity must have a requirement behind it
• FMEA works through failure modes; the newer method uses Action Priority, which puts severity first
• The project proposal brings all deliverables together, traces back to the requirements and includes stop or review conditions

Check your understanding

1. There are 20 requirements and verification activities cover 17 of them. What is the coverage?
2. S = 7, O = 3, D = 4. What is the RPN?
3. Why did the AIAG-VDA Handbook (2019) move from RPN to Action Priority?
4. In the V model, which verification layer is paired with the system requirements?
5. What does a test activity that verifies no requirement indicate?