Reactis helps engineers build better software faster by automating many verification and validation tasks in a model-based design process.

Over the past decade, many engineering organizations have deployed model-based design to address the exploding complexity of embedded control software. In model-based design, executable visual models of embedded control software are developed in advance of system implementation. The models may be used to drive the development of control software, and may also serve as a basis for software and system testing. One benefit of model-based design is that it allows engineers to begin debugging and validation activities at design time, when the cost of detecting and dealing with design defects is much smaller than at the software and system implementation level. Another is that models may be used as a baseline for assessing implementation behavior during system testing and validation. For these reasons, judicious use of modeling can lead to quite dramatic over-all reductions in the cost of control-system development, especially when robust tool support is available.

The Reactis tool suite of Reactive Systems, Inc., substantially enhances the gains organizations realize from model-based design by automating many testing and validation activities. Reactis works with models implemented in the Simulink/Stateflow notation offered by MathWorks. Using Reactis, engineers may:

• generate tests from a model that thoroughly exercise the model (structural testing);
• find runtime errors (e.g. overflow errors, divide-by-zero errors) in a model;
• execute the model and track coverage (e.g. MC/DC);
• perform functional tests to check whether or not a model can violate its requirements;
• use a Reactis test suite as an oracle to check whether code conforms to a model.

Reactis also includes an array of sophisticated model debug features (e.g. breakpoints, scopes, reverse execution). In this paper, we discuss how Reactis and model-based design may be used to automate different verification and validation activities in your software quality assurance process. In particular, we show how the tool may be used to develop more robust models, how it can streamline software and system testing, and how it may be used to support the reverse-engineering of models from legacy code.

A model-based design environment involving Reactis, Simulink and Stateflow is depicted in Figure 1. Reactis contains three core components: Tester, which provides automated test generation from models; Simulator, which enables you to visualize model execution to debug models and track coverage; and Validator, which offers automated checks of Simulink models for violations of user-specified requirements. The remainder of this section describes these components in more detail.

Figure 1: Reactis is used in a model-based design process using Simulink/Stateflow models. Reactis is a standalone application that reads the .mdl/.slx files produced by the MathWorks environment.

Reactis Tester

Figure 2 shows that Reactis Tester offers automatic test generation from Simulink models. The generated test suites provide comprehensive coverage of different test coverage metrics – including the Modified Condition/Decision Coverage (MC/DC) test coverage measure mandated by the US Federal Aviation Administration (FAA) in its DO-178/B guidelines – while at the same time minimizing redundancy in tests. Each test case in a test suite consists of a sequence of inputs fed into the model as well as the responses to those inputs generated by the model.

The automatically generated test data may then be used for a variety of purposes, including the following:

Implementation conformance.

The tests may be applied to source-code implementations of models to ensure conformance with model behavior.

Model testing and debugging.

The tests may be run on the models themselves to detect runtime errors and to study and revise model behavior.

Regression testing of models.

The tests may be run on a new versions of models to flag differing behaviors in new versions.

Figure 2: Reactis Tester automatically generates comprehensive yet compact test suites.

Reverse engineering of models from source.

Tests may be generated from models derived from legacy code in order to check conformance between model and legacy code.

Reactis Tester enables engineers to maximize the effectiveness of testing while reducing the time actually spent on testing.

The structure of a Tester-generated test is shown in Figure 3. A test is constructed by simulating the model and capturing the top-level inputs and outputs. A test may be viewed as a matrix in which each row corresponds to either an inport or outport and each column represents a simulation step. A test suite consists of a set of tests. When running a test suite, the model is reset to its initial state after one test completes and before the next test begins.

Test suites are constructed by simulating a model and recording the input and output values at each step. The model computes the outputs at each step, but several approaches are possible for selecting the input values to drive simulation. The input data could be captured during field testing or constructed manually by an engineer, but these are expensive tasks. Alternatively, the inputs could be generated randomly; however, this approach yields tests with poor coverage.

Figure 3: Structure of a Reactis-generated test.

Reactis Tester employs a patented technique called guided simulation to generate quality input data automatically. The idea behind this approach is to use algorithms and heuristics to automatically generate inputs that cause coverage targets (i.e. model elements that the user wants to ensure are executed at least once) that have not yet been covered to be executed. Reactis currently allows you to track several different classes of coverage targets (also called coverage criteria or coverage metrics). Some of the test coverage metrics supported by Reactis involve only Simulink, some are specific to Stateflow, and the remaining are generic in the sense that they include targets within the Simulink, Stateflow, or C code portions of a model.

Simulink-specific:

Conditional subsystems. Branches of the following blocks: Dead Zone, Logical Operator, MinMax, Multiport Switch, Relational Operator, Saturation, Switch. Lookup tables.

Stateflow-specific:

States. Condition actions. Transition actions. Child State Exit via Parent Transition (CSEPT).

Generic:

Decisions from logic blocks in Simulink, transition segments in Stateflow, or in C code a boolean-valued expression used to determine which execution path to follow. Conditions (the atomic predicates that are the building blocks of decisions). Modified Condition/Decision Coverage (MC/DC) targets. Multiple Condition Coverage (MCC) targets. Boundary Values of top-level inputs.

Reactis Simulator

Reactis Simulator enables you to visualize model execution and debug Simulink models. Simulator’s user interface is similar to those of traditional debuggers from programming languages: it allows you to step through the execution of models by hand as well as set break points. Simulator also supports reverse execution, the replay of tests generated by Reactis Tester, the graphical display of different coverage metrics, the display of data item values, and the capability to fine-tune automatically generated test suites. Stepping between the Simulink, Stateflow, and C code portions of a model is seamless.

Reactis Validator

Reactis Validator performs automated searches of models for violations of user-specified requirements. If Validator finds a violation, it returns a test that leads to the problem. This test may then be executed in Reactis Simulator to gain an understanding of the sequence of events that leads to the problem. Validator enables the early detection of design errors and inconsistencies and reduces the effort required for design reviews. Some checks that may be performed with Validator include the following.

• Will a car’s brake pedal always deactivate the cruise-control?
• Will a plane’s thrust reversers ever engage when the aircraft is airborne?
• Will a medical device ever deliver an unsafe dose of radiation?