Unit testing with optional ports / testing failure conditions
Hello,
I have a component with the following spec:
<ComponentSpec>
<Property name='scalingStyle' type='string' stringlength='10' writable='true' default='log'/>
<Property name='maxValueAfterScale' type='ULong' writable='true' default='255'/>
<Property name='verbose' type='bool' writable='true' default='false'/>
<Port name='in' producer='false' protocol='largeIqStream-prot.xml'/>
<Port name='outULong' producer='true' protocol='largeULongStream-prot.xml' optional='true'/>
<Port name='outUShort' producer='true' protocol='largeUShortStream-prot.xml' optional='true'/>
<Port name='outUChar' producer='true' protocol='largeUCharStream-prot.xml' optional='true'/>
</ComponentSpec>
In order to avoid sending unnecessarily large streams of data, the output port that should be used will depend on the value of maxValueAfterScale (0-255 uCharStream, 255-65,535 uShortStream, 65,535+ uLongStream).
Within the RCC Worker for this component, a check is performed that only the correct output port is connected and RCC_FATAL is returned if this is not the case.
-
Is there a way I can specify within a unit test that I only want certain output ports to be connected? The application file that gets generated has all 3 ports connected with the following test XML:
<Tests UseHDLFileIo='true'>
<Input port='in' script='generate.py'/>
<Output port='outUChar' script='verify.py'/>
<Property name='scalingStyle' values='log, linear, none'/>
<Property name='maxValueAfterScale' values='100, 10000, 100000'/>
</Tests> -
Thinking more generally, how can I test within a unit test that a component fails when it is supposed to, and test the console output it gives in these fail situations.
On 6/1/23 11:01 AM, dwp@md1tech.co.uk wrote:
Hello,
I have a component with the following spec:
|<ComponentSpec>
<Property name='scalingStyle' type='string' stringlength='10'
writable='true' default='log'/>
<Property name='maxValueAfterScale' type='ULong' writable='true'
default='255'/>
<Property name='verbose' type='bool' writable='true' default='false'/>
<Port name='in' producer='false' protocol='largeIqStream-prot.xml'/>
<Port name='outULong' producer='true'
protocol='largeULongStream-prot.xml' optional='true'/>
<Port name='outUShort' producer='true'
protocol='largeUShortStream-prot.xml' optional='true'/>
<Port name='outUChar' producer='true'
protocol='largeUCharStream-prot.xml' optional='true'/>
</ComponentSpec>|
In order to avoid sending unnecessarily large streams of data, the
output port that should be used will depend on the value of
|maxValueAfterScale| (0-255 |uCharStream|, 255-65,535 |uShortStream|,
65,535+ |uLongStream|).
Within the RCC Worker for this component, a check is performed that
only the correct output port is connected and |RCC_FATAL| is returned
if this is not the case.
Is there a way I can specify within a unit test that I only want
certain output ports to be connected? The application file that
gets generated has all 3 ports connected with the following test XML:
|<Tests UseHDLFileIo='true'>
<Input port='in' script='generate.py'/>
<Output port='outUChar' script='verify.py'/>
<Property name='scalingStyle' values='log, linear, none'/>
<Property name='maxValueAfterScale' values='100, 10000, 100000'/>
</Tests>|
|I believe the "testoptional" attribute (for input or output) specifies
that an optional port is disconnected in the test case.
Not a great name I suppose. This is in the documentation.|
Thinking more generally, how can I test within a unit test that a
component fails when it is supposed to, and test the console
output it gives in these fail situations.
Since failure can happen for many reasons, there is no such option (to
specify that ocpirun returns an exit status of non-zero).
Perhaps some more narrowly specified error condition (i.e. the worker
failed explicitly) could be a good option to propose.
The best you could do now is to designate a property that indicated
something like that (and perhaps also issue an EOF to end execution
early), but that sounds clumsy.
discuss mailing list --discuss@lists.opencpi.org
To unsubscribe send an email todiscuss-leave@lists.opencpi.org
Hi James,
Thanks this is exactly what I was looking for. I am running into some strange behaviour however which I assume is a bug.
I have split my test XML into 3 cases: (Used underscores for indentation as spaces get removed when I post)
<Tests UseHDLFileIo='true'>
__<Input port='in' script='generate.py'/>
__<Case>
____<Output port='outUChar' testOptional='true'/>
____<Output port='outUShort' testOptional='true'/>
____<Output port='outULong' script='verify.py'/>
____<Property name='maxValueAfterScale' values='1000000'/>
____<Property name='scalingStyle' values='log, linear, none'/>
__</Case>
__<Case>
____<Output port='outUChar' testOptional='true'/>
____<Output port='outUShort' script='verify.py'/>
____<Output port='outULong' testOptional='true'/>
____<Property name='maxValueAfterScale' values='10000'/>
____<Property name='scalingStyle' values='log, linear'/>
__</Case>
__<Case>
____<Output port='outUChar' script='verify.py'/>
____<Output port='outUShort' testOptional='true'/>
____<Output port='outULong' testOptional='true'/>
____<Property name='maxValueAfterScale' values='100'/>
____<Property name='scalingStyle' values='log, linear'/>
__</Case>
</Tests>
The first two cases work fine, however the final case, when outUChar is connected, does not run and gives the error:
Exiting for exception: Error parsing assembly xml string due to: No instance named "file_write_from_outULong" found
Looking in the generated application XMLs, the first line generated is <application done='file_write_from_outULong'>
whereas for the other cases it is <application done='cmplx_to_magnitude'>.
I have tried changing the order of cases and the order the output ports are listed but it still generates the same.
I just wanted to check I am not doing something wrong before I submit this as an issue on gitlab.
Thanks again for your help.
Dan
Hi,
Turns out that this is not working for the other cases either, when I try to run ocpidev run test the following log is generated:
OCPI_LIBRARY_PATH=../../../lib/rcc:../../../lib/ocl:../../gen/assemblies:$OCPI_CDK_DIR/$OCPI_TOOL_DIR/artifacts:$OCPI_CDK_DIR/$platform/artifacts $OCPI_CDK_DIR/$OCPI_TOOL_DIR/bin/ocpirun -d -v -H -mcmplx_to_magnitude=rcc -wcmplx_to_magnitude=cmplx_to_magnitude -Pcmplx_to_magnitude=ubuntu22_04 --sim-dir=case00.00.cmplx_to_magnitude.rcc.simulation --timeout=3600 --dump-file=case00.00.cmplx_to_magnitude.rcc.props -pfile_write=fileName=case00.00.cmplx_to_magnitude.rcc.outULong.out ../../gen/applications/case00.00.xml
Available containers are: 0: rcc0 [model: rcc os: linux platform: ubuntu22_04]
Exiting for exception: Error parsing assembly xml string due to: No instance for property assignment 'file_write=fileName=case00.00.cmplx_to_magnitude.rcc.outULong.out'
Am I doing something wrong here or is this a bug?
On 6/7/23 7:29 AM, dwp@md1tech.co.uk wrote:
Am I doing something wrong here or is this a bug?
discuss mailing list -- discuss@lists.opencpi.org
To unsubscribe send an email to discuss-leave@lists.opencpi.org
The applications generated by the unit test framework designate one
worker as the "done worker" (or "finished worker"), which is usually the
file writing worker associated with the first output port.
This is explained in the descriptions of the finishPort and
finishedWorkerIsUUT attributes in the documentation.
Since the default finished worker is for the first output port, when
that port is not connected (testOptional), then things won't work.
The "finishPort" attribute is unfortunately global, which means it is
set for all cases, and cannot be set per-case.
Certainly some better error messages would help here, as would the
ability to set "finishport" for each case.
But in fact it appears that the "testOptional" feature does not work
with multiple ports like this in any case.
You are unlucky this week. Thanks for the report - it is clear what
needs fixing.
Jim
Hi Jim,
Okay thank you for the explanation, for now I will write application XMLs that generate each case and manually verify them.
Kind regards,
Dan