F-47 ANS Performance Validation Project

The capstone exercise for SY6301 is a developmental test (DT) campaign on the F-47 Alternative Navigation Suite (ANS). Your team is responsible for designing and executing a statistically defensible evaluation of the upgraded ANS against four governing requirements, then briefing the results in a formal Preliminary Report of Results.

This page is an operational summary. The full project handout is embedded below. The mathematical justification for the analysis methods (correlation time, \(N_\text{eff}\), ECDF for accuracy, OLS drift CI, peak-and-snapback inference, the 27/30 rule) is in Block 9 — Statistical Test Planning.

Background

The F-47 Navigation Computer has been upgraded with an Alternative Navigation Suite (ANS) designed to sustain navigation performance in contested, GPS-denied environments. The ANS augments the baseline GPS-inertial navigation solution with Alternative Navigation (AltNav) aiding sources, including Magnetic Navigation (MagNav), Visual-Aided Navigation (VisNav), and Signals of Opportunity Navigation (SoPNav).

The program office has directed a developmental test evaluation to characterize navigation accuracy, inertial-only drift behavior, and integrity performance under representative operational conditions, and to verify compliance with the four mission-relevant requirements summarized below.

Hardware-in-the-loop (HITL) testing is complete. The HITL campaign measured horizontal-error correlation time at approximately 15 seconds in steady-state operation, ran more than 200 fault events across step-bias and ramp-bias scenarios, and identified single-satellite ramp bias as the most stressing fault case (longest time-to-detect, greatest HMI exposure). DT will focus on this case to bound worst-case fault-detection performance.

System Under Test

The ANS operates in three modes:

Mode	Sensor suite
AllSource	Inertial + GPS + AltNav
AltNav	Inertial + AltNav (GPS denied)
Inertial	Inertial only

Each output stream contains: timestamp, true position (LLH from a separate truth reference), estimated position (LLH), estimated position covariance (NED), and integrity protection levels (HPL, VPL).

Four Governing Requirements

Req	Mode	Threshold	Sample requirement
1	AllSource	\(H_{95} \le 1.0\) m, \(V_{95} \le 5.0\) m	\(N_\text{eff} \ge 300\)
2	AltNav	\(H_{95} \le 5.0\) m, \(V_{95} \le 10.0\) m	\(N_\text{eff} \ge 300\)
3	Inertial	Drift rate \(\le 1.0\) NM/hr (upper 95% CI)	\(T \ge 85\) min
4	AllSource	\(T_D \le 5\) s, \(\text{HMI}_H \le 1\) s, \(\text{HMI}_V \le 1\) s	27 of 30 events

Requirements 1 and 2 are accuracy assessments. The threshold is a tail bound on the 95th-percentile of horizontal and vertical position error. With a steady-state correlation time of 15 s and \(N_\text{eff} \ge 300\), the floor on run duration is 150 minutes. See Block 9 for the derivation.

Requirement 3 is an inertial-drift slope estimate against a 1 NM/hr threshold on the upper 95% confidence bound. Run duration must be at least one full Schuler cycle (~85 minutes); see Block 2 for why.

Requirement 4 is a multi-event fault-detection assessment. Per-event metrics are time-to-detect (\(T_D\)) and HMI exposure (horizontal and vertical separately). Compliance is at least 27 of 30 events meeting all three per-event thresholds simultaneously. The 27-of-30 rule encodes a per-event success probability in the 0.90 to 0.95 range, as shown in Block 9.

Datasets Provided

Four .mat files ship with the course distribution under code/project/. They contain time-stamped truth and estimate streams plus integrity outputs:

File	Mode	Used for
data_AllSource	AllSource	Requirement 1 accuracy
data_AltNav	AltNav	Requirement 2 accuracy
data_Inertial	Inertial	Requirement 3 drift
data_Spoof	AllSource (with injected faults)	Requirement 4 fault detection

Each dataset contains time (s), true position (LLH), estimated position (LLH), covariance (NED), integrity (HPL, VPL), and — for data_Spoof only — fault onset times \(t_0\) for each event.

Critical warning

data_Spoof must not be used for accuracy assessment. The fault-injection events produce episodically elevated errors that inflate the 95th-percentile estimate. Accuracy and fault-detection testing require separate dedicated runs.

Analysis Scripts

Five focused MATLAB scripts ship with the project. Each is self-contained and aligned to one or two requirements. Work through them in order.

Script	Task	Requirement
Project_DataRequest.m	Compute \(N_\text{eff}\) floor, justify run durations	Planning
Project_CorrelationTime.m	Build ACF from data, estimate \(T_\text{corr}\), verify \(N_\text{eff}\)	Planning
Project_AccuracyECDF.m	95th-percentile ECDF of horizontal/vertical error vs bound	1 and 2
Project_InertialDrift.m	OLS slope, 95% CI, NM/hr comparison	3
Project_FaultDetection.m	Infer \(t_D\) via peak-and-snapback, compute HMI exposure, 27/30 count	4

Block 9 walks through the underlying math for each script. The scripts themselves live in code/project/ alongside the datasets.

Deliverable: Preliminary Report of Results

The summative deliverable is a formal Preliminary Report of Results (PRR) briefing that consolidates compliance evidence on all four requirements. The class operates as a single DT team with sub-teams aligned to individual requirements. The briefing is evaluated against a 20-point rubric covering problem framing, statistical methodology (correlation-time justification, \(N_\text{eff}\) accounting, choice of test statistic), per-requirement compliance comparisons, traceability between data and conclusion, and an executive recommendation that ties the technical findings back to operational mission impact.

Detailed grading criteria are in the project handout below.

Project Handout (PDF)

Download the project handout PDF

How to Use This Section

1. Read this overview page.

2. Read Block 9 — Statistical Test Planning for the F-47 ANS to understand the math behind every analysis decision.

3. Open the project handout PDF (above) for the full operational specification, deliverable expectations, and rubric.

4. Use the analysis scripts in code/project/ as your starting templates. Each maps to one requirement and is self-contained.

5. Build your DT team’s sub-team assignments around the four requirements, plan minimum run durations from the \(N_\text{eff}\) analysis, and brief the result in the PRR.