David,

Thanks for the feedback. Looking it over, I can see some places where it
can be improved because it seems to slurp up the whole results file without
iterating over it with something like numpy.memmap. At this time, we don't
have the resources to dig into it, but I've assigned it bug #AV-5098.

Thanks,

Aaron

From: discuss discuss-bounces@lists.opencpi.org on behalf of David
Banks dbanks@geontech.com
Sent: Thursday, January 17, 2019 12:00:47 PM
To: discuss@lists.opencpi.org
Subject: [Discuss OpenCPI] data_src.test verification using up 24GB RAM
and failing

On my system with 24GB of RAM, data_src.test (from release_1.4)
verification fails for case00.02 with the following python error:

Executing case case00.02 using worker data_src.hdl...
Execution succeeded, time was 00:01:07 (67 seconds).
Verifying case case00.02 for worker data_src.hdl using script on output
file: case00.02.data_src.hdl.out.out
PASS: output file length >= 1 samples for 'num_samples' property value
= -1
Traceback (most recent call last):
File "../../verify.py", line 187, in <module>
main()
File "../../verify.py", line 184, in main
test_odata_file_binary_contents_are_as_expected()
File "../../verify.py", line 115, in
test_odata_file_binary_contents_are_as_expected
data = np.remainder(np.arange(len(odata)), 2**DATA_BIT_WIDTH_p)
MemoryError

• Verification for port out: FAILED*
  make: *** [run] Error 1

The first two cases succeed and pass verification. I monitored my system
resources while it was running case00.02 and saw that the python3 process
used up ~100% of my system's 24GB of memory.

David Banks
dbanks@geontech.com
Geon Technologies, LLC

David,

Thanks for the feedback. Looking it over, I can see some places where it
can be improved because it seems to slurp up the whole results file without
iterating over it with something like numpy.memmap. At this time, we don't
have the resources to dig into it, but I've assigned it bug #AV-5098.

Thanks,

Aaron

From: discuss discuss-bounces@lists.opencpi.org on behalf of David
Banks dbanks@geontech.com
Sent: Thursday, January 17, 2019 12:00:47 PM
To: discuss@lists.opencpi.org
Subject: [Discuss OpenCPI] data_src.test verification using up 24GB RAM
and failing

On my system with 24GB of RAM, data_src.test (from release_1.4)
verification fails for case00.02 with the following python error:

Executing case case00.02 using worker data_src.hdl...
Execution succeeded, time was 00:01:07 (67 seconds).
Verifying case case00.02 for worker data_src.hdl using script on output
file: case00.02.data_src.hdl.out.out
PASS: output file length >= 1 samples for 'num_samples' property value
= -1
Traceback (most recent call last):
File "../../verify.py", line 187, in <module>
main()
File "../../verify.py", line 184, in main
test_odata_file_binary_contents_are_as_expected()
File "../../verify.py", line 115, in
test_odata_file_binary_contents_are_as_expected
data = np.remainder(np.arange(len(odata)), 2**DATA_BIT_WIDTH_p)
MemoryError

• Verification for port out: FAILED*
  make: *** [run] Error 1

The first two cases succeed and pass verification. I monitored my system
resources while it was running case00.02 and saw that the python3 process
used up ~100% of my system's 24GB of memory.

David Banks
dbanks@geontech.com
Geon Technologies, LLC

Hi David,

Multiple issues with this worker were identified after the 1.4 release.
While they didn't not manifest themselves in the failures you are
describing, it could be worth trying the fixes. The fixes can be found in
the develop branch.

Thanks,

Brandon

From: discuss discuss-bounces@lists.opencpi.org on behalf of David
Banks dbanks@geontech.com
Sent: Friday, January 18, 2019 8:47:20 AM
To: discuss@lists.opencpi.org
Subject: Re: [Discuss OpenCPI] data_src.test verification using up 24GB
RAM and failing

No problem at all. I just want to make sure someone is aware of this
bug/limitation of that unit test. The only other problem I have experienced
with OpenCPI Component Unit Tests is that zero_pad.test case00.01 will hang
indefinitely during execution on hardware. Has anyone seen this behavior
before?

Thanks,
David

On Fri, Jan 18, 2019 at 6:59 AM Marasco, Aaron <
aaron.marasco@bia-boeing.com>
wrote:

David,

Thanks for the feedback. Looking it over, I can see some places where
it
can be improved because it seems to slurp up the whole results file
without
iterating over it with something like numpy.memmap. At this time, we
don't
have the resources to dig into it, but I've assigned it bug #AV-5098.

Thanks,

Aaron

---

From: discuss discuss-bounces@lists.opencpi.org on behalf of David
Banks dbanks@geontech.com
Sent: Thursday, January 17, 2019 12:00:47 PM
To: discuss@lists.opencpi.org
Subject: [Discuss OpenCPI] data_src.test verification using up 24GB RAM
and failing

On my system with 24GB of RAM, data_src.test (from release_1.4)
verification fails for case00.02 with the following python error:

Executing case case00.02 using worker data_src.hdl...
Execution succeeded, time was 00:01:07 (67 seconds).
Verifying case case00.02 for worker data_src.hdl using script on output
file: case00.02.data_src.hdl.out.out
PASS: output file length >= 1 samples for 'num_samples' property
value
= -1
Traceback (most recent call last):
File "../../verify.py", line 187, in <module>
main()
File "../../verify.py", line 184, in main
test_odata_file_binary_contents_are_as_expected()
File "../../verify.py", line 115, in
test_odata_file_binary_contents_are_as_expected
data = np.remainder(np.arange(len(odata)), 2**DATA_BIT_WIDTH_p)
MemoryError

• Verification for port out: FAILED*
  make: *** [run] Error 1

The first two cases succeed and pass verification. I monitored my system
resources while it was running case00.02 and saw that the python3 process
used up ~100% of my system's 24GB of memory.

David Banks
dbanks@geontech.com
Geon Technologies, LLC