Hi,
Thanks to all for your replies. I will use Gabor's mail to gather my
comments below.
I believe that you treat the wrong problem. It is exactly the same test-controllability problem that you would face with "real" SUT. How do you trigger an expected behavior from an "real" SUT if the test case is supposed to start by waiting for a message instead of sending a stimulus first?
Yes, this is exactly our problem, how do you tell the SUT or the tester that the test system is ready ?!
You *must* figure out how would you solve that real problem, then include the very same trigger mechanism into your SDL(-RT) model and your original problem will disappear.
Imagine a bunch of PTCs exchanging messages, and mapping or unmapping ports before actually starting the test case... How can a tool figure out when the test is ready ?! Only the guy who wrote the test case knows
for sure.
I know that that my suggestion "complicates" your life in the sense that you face yet another challenging problem instead of the original one, but the benefit would two-fold: you could reliably test either your abstract SDL model or any concrete "hardware" implementation.
This is a very good point because we actually do not have this problem when co-simulating SDL and TTCN-3 together and let me explain why. When co-simulating we get full control on the overall architecture of both the system and the test cases. Because the simulator is the scheduler, we actually run the Test case first and then the SDL system. If the Test case sent a message in the meantime, the simulator will hold it and re-route as soon as the SDL system is executed. This is not possible with generated code on a Linux target for example.
We have had long discussion internally regarding that issue and this is what came out:
It looks like this synchronization problem is raised because the System interface is not static. Each test case can define its own system interface. When running one test case after another, the system interface actually disappears with the ending test case and is created again with the new test case. So what happens if theSUT send messages before the system interface is defined or in between two test cases is undefined. This is very strange because the system interface on a real tester does not change and should actually be defined statically. The test cases could map or unmap to that static interface but the overall system interface does not change. The introduction of a dynamic system interface has introduced a synchronization problem the language does not address in its current version.
Considering the language as it is today, the tester writing the test case should somehow be able to indicate the test case is all set and ready with a standard TTCN-3 statement.
Nevertheless, you might have workarounds:
1. The stupid solution #1: use artificial delays and timers. I.e., trigger the SUT somehow from the test case (e.g., via a separate test coordination interface (port) that should be mappable from the beginning), then wait for a given delay before attempting to map the real test port. It is "stupid", since the solution is not robust with respect to the delay interval, i.e., your test case might wait too short time ("test case too fast").
2. The stupid solution #2: try to launch the SUT from within the control part, i.e., before launching the test case. It is "stupid", since the solution is not robust with respect to the delay interval, i.e., your test case might fire up too late and perform the mapping *after* the SUT has already sent the message ("test case too slow").
Frankly we would not consider these as proper industrial solutions.
For a robust solution you *must* remove the dependency on the startup time of either the SUT or the test case proper, i.e., you *must* explicitly perform the synch between the two peers. See my suggestion at the beginning of my mail.
The action statement solution you mentioned looks like a good workaround for handling this synchronization issue but we see major drawbacks in that solution:
This is a tool specific solution for a language problem.
This implies existing test suites will have to be modified to run.
I don't think that is what a user wants if it is a standard test case.
kr,
Emmanuel