Metrology at Production Speed: Balancing Uncertainty, Test Time and Acceptance Decisions
Modern production test systems operate under conditions that are very different from those of a metrology laboratory. Measurement accuracy remains important, but it has to be achieved together with repeatable operation, controlled test time, and reliable acceptance decisions. Metrology News spoke with Tatiana Krasik, an independent researcher and measurement engineer whose work includes automated test equipment, RF and microwave measurements, calibration, and production testing at frequencies up to 200 GHz.
Q: What changes when laboratory measurement methods are transferred into production?
A: A laboratory method is usually developed around the measurement itself. In production, the measurement is one operation inside a larger process, so test time, equipment availability and repeatability become design parameters along with uncertainty. A procedure that works well for ten devices may be unsuitable when the same operation has to be performed several thousand times. Long stabilization, repeated reconnections or unnecessary averaging accumulate into significant production time. The first task is therefore to define the uncertainty that can actually affect the acceptance decision and then determine which operations are required to keep that uncertainty under control. If a measurement can be made in two seconds with sufficient confidence, there is little reason to spend ten seconds obtaining a result that does not change the pass/fail decision. The production method has to be designed for the required decision, not copied directly from a laboratory procedure.
Q: What problems do you see when laboratory procedures are automated without modification?
A: The usual result is an oversized test sequence. Every measurement, check and calibration step from the laboratory procedure is transferred into software because it already exists, not because it is required in production. Test time increases, the system becomes harder to maintain, and additional switching or reconnection operations introduce their own errors. I normally start with the product specification and work backward. For each parameter, I need to know the acceptance limit, expected process variation, required measurement uncertainty and the cost of a wrong decision. After that, the sequence can be reduced to the operations that actually contribute to the decision. Some parameters may need several measurements. Others need only one. A verification measurement may make sense only near the specification limit. Calibration should be applied to the part of the path where it controls a significant error source. Automating a procedure is not the same as designing an automated test.
Q: Why does measurement uncertainty matter in a pass/fail system?
A: The tester compares a measured value with a limit, but the measured value is not the true value of the parameter. If the result is well inside the allowable range, uncertainty may have little effect on the decision. Near the specification limit, it can determine whether the decision is reliable. For example, if the upper limit is 10.0 and the measured value is 9.9, calling the unit compliant without considering the uncertainty of that measurement may be incorrect. The same effect works in the other direction and can reject a compliant unit. In a production line this is not an isolated metrology problem; it changes yield, rework volume and potentially the number of nonconforming units released. Acceptance limits therefore have to be considered together with measurement uncertainty. Guard bands are useful when the risk associated with a decision requires the acceptance boundary to be different from the product specification boundary.
Q: If repeated measurements reduce uncertainty, why not simply take more of them?
A: Averaging reduces random variation. It does not remove calibration error, fixture error, residual path-loss error, reference-standard uncertainty or a stable systematic offset. If random variation is dominant, the first few additional measurements can noticeably improve repeatability. As the number of measurements increases, that contribution becomes smaller and the remaining uncertainty is determined by components that averaging does not reduce. At that point more measurements mostly add test time. This can be seen directly from an uncertainty budget: once the repeatability component is small compared with the other contributors, increasing the sample count produces very little change in total uncertainty. The number of repeated measurements should therefore be justified by the uncertainty target and test-time requirement, rather than selected as a fixed conservative value.
Q: How does calibration architecture affect production testing?
A: In an RF test system, calibration is associated with a measurement path, not only with the instrument. That path may contain switches, cables, adapters, fixtures and connectors, and any of these elements can change. If the system is calibrated only as one complete path, a deviation gives little information about where the problem occurred. Engineers then spend time recalibrating or replacing parts of the setup that may still be stable. Where the hardware allows it, I prefer to provide intermediate verification points or other means of separating the path into sections. A verification result can then indicate whether the change is located before or after a particular point. This shortens troubleshooting and can avoid a full system recalibration after a local change. The calibration architecture should therefore be considered when the tester is designed, not added after the hardware has already been built.
Q: What changes at microwave and millimeter-wave frequencies?
A: The measurement path becomes increasingly sensitive to components that are often treated as secondary at lower frequencies. Cable loss is higher, connector condition matters more, fixture repeatability becomes more difficult to maintain, and small mechanical changes can produce measurable electrical changes. The reference plane also becomes critical. If a system is calibrated at the instrument port but the DUT is connected through switches, cables and a fixture, those elements have to be included in the error analysis or compensated separately. Repeated mating cycles create another production-specific problem because connector and fixture characteristics may change with wear. At these frequencies I treat the complete path from the calibrated reference plane to the DUT interface as part of the measurement system. Instrument accuracy alone does not describe the accuracy of the production measurement.
Q: At what stage should the measurement engineer be involved in defining production pass/fail rules?
A: The measurement engineer should be involved before the acceptance limits are implemented in the test software. A pass/fail rule depends on more than the product specification because the measurement system has its own uncertainty, repeatability and possible bias. If the acceptance rule is defined first and the measurement capability is evaluated later, the result may be a test that cannot reliably separate conforming and nonconforming units near the limit. The required uncertainty, guard band if one is needed, number of repeated measurements and handling of borderline results should be considered together. This also affects hardware choices because the required decision confidence may determine the instrument, fixture, switching path and calibration method. Once production is running, changing the decision rule can affect yield and rework, so these questions are cheaper to solve during test development than after release.
Q: Where can AI actually be useful in production metrology?
A: The practical use is in analyzing data that the test system already produces. A production tester can collect thousands of results from the same measurement path, together with calibration checks, temperatures, station identifiers and diagnostic data. Most conventional software uses an individual result to make a pass/fail decision and does little with the history. Statistical models or machine-learning methods can use that history to detect drift, changes in distributions or combinations of parameters that are unusual for a particular station or fixture. For example, if several RF measurements gradually shift in the same direction while the product process remains stable, that may indicate a change in the test path before any individual result exceeds its limit. The model can flag the condition, but it still cannot determine measurement validity by itself. Training data obtained from an unstable or incorrectly calibrated system will reproduce the same problem at a larger scale. Traceability, uncertainty and verification remain metrology tasks; AI can help identify where to investigate.
Q: How should production measurement systems develop from here?
A: Production measurement systems can make better use of information about the state of the measurement path. A tester usually records whether the DUT passed, while calibration checks, verification data and long term stability are often handled separately. Bringing those data together makes it easier to recognize when the tester itself is changing. The test sequence can also be made less rigid. A result close to a decision boundary may justify an additional measurement, while repeating the same operation on a result far from the limit may add no useful information. The same approach can be applied to maintenance: a local verification failure should lead to investigation of the affected section rather than automatic recalibration of the complete system. Monitoring the measurement path over time helps detect drift before it affects yield. Verification data and historical trends can also reduce unnecessary recalibration and repeated testing.
Tatiana Krasik is an IEEE Senior Member, holds a Candidate of Technical Sciences degree (Ph.D. equivalent), and is the author and co-author of technical and scientific publications in measurement engineering and related fields.
