Testability Analysis · Chapter 4

Worked Example

The method applied end-to-end on a concrete system, with numbers.

Consider the down-converter chain of a satellite ground station: eight replaceable items whose failure modes come straight from the FMECA worksheet, with each mode's rate already computed there as λm = α · λp. Every value is an illustrative teaching figure; the method is the standard's.

The fault list is the FMECA's mode list

The prediction gives an item its rate, the FMECA divides that rate into modes through α, and the dependency matrix takes one column per mode. What goes back the other way is the FMECA's detection column, filled in with something computed rather than asserted.
The prediction gives an item its rate, the FMECA divides that rate into modes through α, and the dependency matrix takes one column per mode. What goes back the other way is the FMECA's detection column, filled in with something computed rather than asserted.
ModeItemλpαλm
M1LNA · gain collapse8.00.705.6
M2LNA · noise rise, gain intact8.00.302.4
M3Image filter · insertion loss rise1.51.001.5
M4Mixer · conversion loss rise6.01.006.0
M5Local oscillator · unlock22.00.6514.3
M6Local oscillator · phase noise rise22.00.357.7
M7IF amplifier · gain loss12.01.0012.0
M8Power supply · rail out of tolerance30.01.0030.0
M9Controller · halt18.01.0018.0
M10Output isolator · slow degradation5.01.005.0
Σ λm102.5

The α column is what makes this an analysis rather than a checklist. The LNA fails at 8.0 per 10⁶ hours, and 70 per cent of that is a gain collapse the IF level check sees while 30 per cent is a noise rise it does not. Analysed as an item, the LNA is "detected". Analysed as modes, 2.4 per 10⁶ hours of it is not.

The matrix

Ten modes, five tests. Working from the FMECA's modes rather than its items is what keeps the LNA's noise rise and the oscillator's phase noise visible as separate columns.
Ten modes, five tests. Working from the FMECA's modes rather than its items is what keeps the LNA's noise rise and the oscillator's phase noise visible as separate columns.

Five measurements are available: RF input power (T1), LO lock detect (T2), IF output level (T3) and the controller watchdog (T4) online, plus a noise-figure check (T5) that needs the chain off line.

FDR, FIR and the false alarm ratio, computed

The columns sorted under the online tests alone. Four modes isolate, three share a signature, and three produce no response at all.
The columns sorted under the online tests alone. Four modes isolate, three share a signature, and three produce no response at all.

Fault detection rate, over the online tests. Three modes have an all-zero column, worth 2.4 + 7.7 + 5.0 = 15.1 per 10⁶ h:

FDR = (102.5 − 15.1) / 102.5 = 87.4 / 102.5 = 85.3 per cent

Fault isolation rate, as a share of what is detected. M3, M4 and M7 share the signature 0 0 1 0, three different items at 19.5 per 10⁶ h:

FIR(1) = (87.4 − 19.5) / 87.4 = 67.9 / 87.4 = 77.7 per cent

Because the three ambiguous modes belong to three separate items, FIR(≤2) is the same 77.7 per cent: a group of three is not resolved by allowing two.

False alarm ratio. This is the one the matrix cannot give, because it is about what the tests do when nothing has failed. Each test carries its own rate of spurious indications, from threshold margin, transients and its own hardware. The four online tests are what configuration A runs; the last three are priced here so the configurations below reproduce:

TestFalse indications per 10⁶ h
T1 · RF input power0.8
T2 · LO lock1.2
T3 · IF output level3.0
T4 · Controller watchdog4.0
The four online tests9.0
T5 · Off-line noise figure0.4
T6 · Mixer bias monitor2.5
T7 · Phase-noise check1.0

FARr = FA / (FA + TA) = 9.0 / (9.0 + 87.4) = 9.3 per cent

About one alarm in eleven is raised by nothing at all. That figure decides whether the other two are acted on, and no amount of dependency modelling produces it.

Adding a test, and the result nobody expects

Four configurations. B detects everything and isolates less of it, because the three modes the off-line check newly finds arrive as one symptom and enter the isolation denominator as a group.
Four configurations. B detects everything and isolates less of it, because the three modes the off-line check newly finds arrive as one symptom and enter the isolation denominator as a group.
ConfigurationFDRFIR(1)FA per 10⁶ hFARr
A · online BIT, four tests85.3%77.7%9.09.3%
B · + off-line noise figure (T5)100%66.2%9.48.4%
C · + mixer bias monitor (T6)100%72.1%11.910.4%
D · + phase-noise check (T7)100%79.6%12.911.2%

Configuration B is the lesson. The off-line noise-figure check responds to exactly the three modes nothing else sees, so detection goes to 100 per cent. But it responds to all three identically, so they arrive as a new three-way ambiguity group, and since isolation is a share of detected faults, FIR(1) falls from 77.7 to 66.2 per cent. The design strictly improved and one of its headline numbers went down. A programme managing to a single isolation target would read that as a regression and remove the test.

Notice also that B's FARr improves, from 9.3 to 8.4 per cent, even though a test was added and false alarms went up with it. T5 contributes 0.4 spurious indications per 10⁶ h, taking FA from 9.0 to 9.4, while the 15.1 it newly detects goes into the same denominator as true alarms:

FARr = 9.4 / (9.4 + 102.5) = 8.4 per cent

Two of the three figures moved for reasons that have nothing to do with what a reader would infer from them, which is the argument for quoting all three together with their bases.

C and D then buy isolation: the mixer bias monitor splits the mixer out of the first group, the phase-noise check splits the oscillator out of the second, and each brings its own false-alarm contribution, taking FA to 11.9 and then to 12.9, so FARr climbs from 8.4 to 11.2 per cent as isolation climbs from 66.2 to 79.6. That trade is the real design decision, and it is invisible unless all three are computed together.

What it costs at the bench

Removals per 1,000 operating hours. Configuration B finds every fault and sends more good modules to stores than A did, because what it newly detects it cannot name.
Removals per 1,000 operating hours. Configuration B finds every fault and sends more good modules to stores than A did, because what it newly detects it cannot name.

A group of n costs (n+1)/2 removals per fault:

ConfigurationRemovals per 1,000 hFaults presentUnnecessary
A0.1070.08718.2%
B0.1370.10225.2%
C0.1240.10217.6%
D0.1130.1029.3%

B is worse at the bench than A in absolute terms: it pulls more modules. It is still the right change, because A was leaving 15.1 per 10⁶ hours of degradation in service undetected, and a slow noise rise on a receive chain is a customer complaint rather than an alarm. Detection and isolation have to be bought together, and the arithmetic above is what shows the size of the second bill.

The strategy

The sequence read off the same ten columns. Starting with the test that responds to six of the ten modes halves the problem in one measurement.
The sequence read off the same ten columns. Starting with the test that responds to six of the ten modes halves the problem in one measurement.

What goes back to the FMECA

The detection column of the worksheet started as an assertion and comes back as a result:

ModeDetection, as recorded now
M1, M5, M8, M9Detected online and isolated to one item
M3, M4, M7Detected online, isolated to a group of three
M2, M6, M10Not detected online; found by the noise-figure check at the scheduled visit

Those last three rows are what a safety or availability argument needs, because their exposure is not the mission but the interval between off-line checks. A worksheet whose detection column says "BIT" against all ten would have hidden that entirely.


Want to see this on a live system model? Request a walkthrough.