Simulation Validation Using Test Data

In simulation, real world data is not always available for validation, or anchoring. Often we must rely on the results of live tests, e.g., flight tests. In the execution of "Results" Validation, we must consider the relationship between the test and M&S environments.

While test is probably the highest fidelity representation of the prototype system, limitations in the test program usually will not allow exercise of the system in its true production configuration or through out its complete envelope. The portion of production environment exercised by the test is depicted in the circles in the Venn diagram in Figure 1 (proportions are for illustration only - no scale is intended).

From Figure 1 we see that some functionality (data) of the test are representative of the production system while others are not. Now consider a simulation of both the test (by a HWIL simulator) and of the production system as shown in the rectangles of Figure 1. Simulation of each environment/configuration of is not an exact representation. The differences between the test and HWIL simulation as well as the difference between the production system and the system simulation must be accounted for.

Figure 1: Simulation of the Test and Production Systems

Two points are worth noting from the above figure. First, while the test may be a higher fidelity representation of the prototype, only some of the functionality applies to the production system. Secondly, the simulation of the system/test is not exact. It does not cover all of the behaviors and also may provide outputs that would not be allowed by the actual configurations. Therefore, care must be taken in the selection of the data that will be used for Results Validation.
The Results Validation Process must define the “accuracy” of the data and determine which data can be used for the purpose of ensuring the fidelity of the M&S. In general, the following assumptions are made for the Results Validation process:

1. The tests are simulations of the prototype (as is) system.

2. The test is the most accurate representation of the prototype system.

3. Once validated by test, the HWIL simulation is the next most accurate simulation of the prototype system.

4. Once validated, the digital simulation follows the HWIL simulator as a depiction of the prototype system.

5. Once validated, the digital simulation is the best representation of the production system in its’ intended environment.