Multiple ad9361_config_proxy Instances
I've been able to build my FPGA image with multiple AD9361's connected.
Now I'm trying to actually interact with them individually and I believe I
have run into a bug with how the ad9361_config_proxy is being initialized.
I am modifying the ad9361_config_proxy_test to instantiate both of my front
ends and all supporting proxies. I'm giving them unique names and working
with their names directly. I am definitely initializing each of the
individual front ends separately, which is great. If I disable one of
them, the appropriate ad9361_init() call fails.
Unfortunately, it seems that whichever was initialized second is the only
one I can access. In my program, I setup 4 properties:
- afe0_config_proxy, temperature
- afe1_config_proxy, temperature
- afe0_config_proxy, rx_lo_freq
- afe1_config_proxy, rx_lo_freq
I then loop through printing all 4 items to a line. The temperatures and
frequencies for both are always the same. When I run the program with
OCPI_LOG_LEVEL=10, I notice the address printed around the No-OS API calls
switches after the first initialization. Here are the two initialization
functions from my log:
OCPI(10:275.0635): Setting the default value of property 'ad9361_init' of
instance 'afe0_config_proxy'
OCPI(10:275.0635): set8RegisterOffset 0xb3059000 24 1
OCPI(10:275.0637): afe0_config_proxy: No-OS API call: ad9361_init(...)
OCPI(10:275.0637): get8RegisterOffset 0xb3059000 12
OCPI(10:276.0305): Setting the default value of property 'ad9361_init' of
instance 'afe1_config_proxy'
OCPI(10:276.0305): set8RegisterOffset 0xb3559000 24 1
OCPI(10:276.0307): afe1_config_proxy: No-OS API call: ad9361_init(...)
OCPI(10:276.0307): get8RegisterOffset 0xb3559000 12
Once afe1_config_proxy has been initialized, the 0xb3059000 address never
shows up. I'd expect, when querying the values from the device, they
should show up. Here are the 4 values being queried:
OCPI(10:283.0173): afe0_config_proxy: No-OS API call:
ad9361_get_temperature(...)
OCPI(10:283.0173): getBytesRegisterOffset 0xb3559000 45 sz 1
OCPI(10:283.0173): afe0_config_proxy: No-OS API call:
ad9361_get_rx_lo_freq(...)
OCPI(10:283.0173): getBytesRegisterOffset 0xb3559000 25d sz 1
OCPI(10:283.0174): afe1_config_proxy: No-OS API call:
ad9361_get_temperature(...)
OCPI(10:283.0174): getBytesRegisterOffset 0xb3559000 45 sz 1
OCPI(10:283.0174): afe1_config_proxy: No-OS API call:
ad9361_get_rx_lo_freq(...)
OCPI(10:283.0175): getBytesRegisterOffset 0xb3559000 25d sz 1
So something seems to be overwriting the initial 0xb3059000 address with
0xb3559000 in the afe0_config_proxy even though it should only be
associated with the afe1_config_proxy.
I didn't see anything obvious in the ad9361_config_proxy.cc but it's also
not the easiest file to read.
Does anyone familiar with that file/subsystem have any insights?
Thanks,
Brian
I've been unable to recreate this issue in xsim, I'll see if there's a way
to emulate this scenario on hardware (we don't have a physical device with
2 AD9361s to test this). You may well have found a bug with the device
proxy worker memory mapping.
---------- Forwarded message ---------
From: Brian Padalino bpadalino@gmail.com
Date: Thu, Feb 20, 2020 at 3:44 PM
Subject: [Discuss OpenCPI] Multiple ad9361_config_proxy Instances
To: discuss@lists.opencpi.org
I've been able to build my FPGA image with multiple AD9361's connected.
Now I'm trying to actually interact with them individually and I believe I
have run into a bug with how the ad9361_config_proxy is being initialized.
I am modifying the ad9361_config_proxy_test to instantiate both of my front
ends and all supporting proxies. I'm giving them unique names and working
with their names directly. I am definitely initializing each of the
individual front ends separately, which is great. If I disable one of
them, the appropriate ad9361_init() call fails.
Unfortunately, it seems that whichever was initialized second is the only
one I can access. In my program, I setup 4 properties:
- afe0_config_proxy, temperature
- afe1_config_proxy, temperature
- afe0_config_proxy, rx_lo_freq
- afe1_config_proxy, rx_lo_freq
I then loop through printing all 4 items to a line. The temperatures and
frequencies for both are always the same. When I run the program with
OCPI_LOG_LEVEL=10, I notice the address printed around the No-OS API calls
switches after the first initialization. Here are the two initialization
functions from my log:
OCPI(10:275.0635): Setting the default value of property 'ad9361_init' of
instance 'afe0_config_proxy'
OCPI(10:275.0635): set8RegisterOffset 0xb3059000 24 1
OCPI(10:275.0637): afe0_config_proxy: No-OS API call: ad9361_init(...)
OCPI(10:275.0637): get8RegisterOffset 0xb3059000 12
OCPI(10:276.0305): Setting the default value of property 'ad9361_init' of
instance 'afe1_config_proxy'
OCPI(10:276.0305): set8RegisterOffset 0xb3559000 24 1
OCPI(10:276.0307): afe1_config_proxy: No-OS API call: ad9361_init(...)
OCPI(10:276.0307): get8RegisterOffset 0xb3559000 12
Once afe1_config_proxy has been initialized, the 0xb3059000 address never
shows up. I'd expect, when querying the values from the device, they
should show up. Here are the 4 values being queried:
OCPI(10:283.0173): afe0_config_proxy: No-OS API call:
ad9361_get_temperature(...)
OCPI(10:283.0173): getBytesRegisterOffset 0xb3559000 45 sz 1
OCPI(10:283.0173): afe0_config_proxy: No-OS API call:
ad9361_get_rx_lo_freq(...)
OCPI(10:283.0173): getBytesRegisterOffset 0xb3559000 25d sz 1
OCPI(10:283.0174): afe1_config_proxy: No-OS API call:
ad9361_get_temperature(...)
OCPI(10:283.0174): getBytesRegisterOffset 0xb3559000 45 sz 1
OCPI(10:283.0174): afe1_config_proxy: No-OS API call:
ad9361_get_rx_lo_freq(...)
OCPI(10:283.0175): getBytesRegisterOffset 0xb3559000 25d sz 1
So something seems to be overwriting the initial 0xb3059000 address with
0xb3559000 in the afe0_config_proxy even though it should only be
associated with the afe1_config_proxy.
I didn't see anything obvious in the ad9361_config_proxy.cc but it's also
not the easiest file to read.
Does anyone familiar with that file/subsystem have any insights?
Thanks,
Brian
On Mon, Feb 24, 2020 at 10:48 AM Davis Hoover dhoover@geontech.com wrote:
I've been unable to recreate this issue in xsim, I'll see if there's a way
to emulate this scenario on hardware (we don't have a physical device with
2 AD9361s to test this). You may well have found a bug with the device
proxy worker memory mapping.
Just to be clear, this seems to be an RCC issue and not an HDL issue.
I believe the HDL is doing exactly what it should be doing, but the proxy
worker on the RCC/ARM side seems to want to point to the last initialized
device.
If you need me to run some commands to help figure this out, please let me
know. I can provide logs.
Thanks,
Brian
I'm guessing this is an issue with how the framework maps the HDL
slave's memory as made available to the RCC proxy. I know there are
some differences in the C++ classes for management of simulator vs
FPGAs, and it seems I need to exercise the latter to recreate the
problem.
It may be helpful to look at your application XML.
---------- Forwarded message ---------
From: Brian Padalino bpadalino@gmail.com
Date: Mon, Feb 24, 2020 at 10:53 AM
Subject: Re: [Discuss OpenCPI] Multiple ad9361_config_proxy Instances
To: Davis Hoover dhoover@geontech.com
Cc: discuss@lists.opencpi.org
On Mon, Feb 24, 2020 at 10:48 AM Davis Hoover dhoover@geontech.com wrote:
I've been unable to recreate this issue in xsim, I'll see if there's a way
to emulate this scenario on hardware (we don't have a physical device with
2 AD9361s to test this). You may well have found a bug with the device
proxy worker memory mapping.
Just to be clear, this seems to be an RCC issue and not an HDL issue.
I believe the HDL is doing exactly what it should be doing, but the
proxy worker on the RCC/ARM side seems to want to point to the last
initialized device.
If you need me to run some commands to help figure this out, please
let me know. I can provide logs.
Thanks,
Brian
On Mon, Feb 24, 2020 at 11:20 AM Davis Hoover dhoover@geontech.com wrote:
I'm guessing this is an issue with how the framework maps the HDL
slave's memory as made available to the RCC proxy. I know there are
some differences in the C++ classes for management of simulator vs
FPGAs, and it seems I need to exercise the latter to recreate the
problem.
It may be helpful to look at your application XML.
The application XML is very simple. I've pasted it below.
<!-- The ad9361_dual_temp application xml file --> <Application> <Instance Component='ocpi.assets.devices.ad9361_spi' Name='afe0_spi'/> <Instance Component='ocpi.assets.devices.ad9361_spi' Name='afe1_spi'/><Instance Component='ocpi.assets.devices.ad9361_config'
Name='afe0_config'/>
<Instance Component='ocpi.assets.devices.ad9361_config'
Name='afe1_config'/>
<Instance Component='ocpi.assets.devices.ad9361_config_proxy'
Slave='afe0_config' Name='afe0_config_proxy'/>
<Instance Component='ocpi.assets.devices.ad9361_config_proxy'
Slave='afe1_config' Name='afe1_config_proxy'/>
</Application>
I have an OCPI_LOG_LEVEL=10 log that I can scrub and attach as well, but
the relevant stuff has already been said in the initial e-mail.
Let me know if there is anything else I can provide to help debug.
Thanks,
Brian
On Mon, Feb 24, 2020 at 11:30 AM Brian Padalino bpadalino@gmail.com wrote:
On Mon, Feb 24, 2020 at 11:20 AM Davis Hoover dhoover@geontech.com
wrote:
I'm guessing this is an issue with how the framework maps the HDL
slave's memory as made available to the RCC proxy. I know there are
some differences in the C++ classes for management of simulator vs
FPGAs, and it seems I need to exercise the latter to recreate the
problem.
It may be helpful to look at your application XML.
I've finally figured out and solved this issue, and it had nothing to do
with the mapping in the HDL. It was a very silly RCC bug that took me ages
to find.
The problem is very straight forward and involves a few files:
- The ad9361_config_proxy.cc calls spi_init() on line 849 which is in
ad9361_platform.cc - spi_init() assigns the pointer to an instance variable
of OCPI::RCC::RCCUserSlave* called slave - slave is used whenever spi_write_then_read() is used, and the struct
spi_device* is not used - Someone planned to use it, since struct spi_device has a device type
which has a void* called slave in it
For my case, each time the initialization of the ad9361_config_proxy was
happening (twice - once for each AFE), spi_init() was being called and kept
re-assigning the slave pointer, even though each ad9361_config_proxy is
separate and have different slaves.
My modification was to assign ad9361_rf_phy->spi->dev.slave after this line:
... and modify the spi_write_then_read() function to:
- check if spi is null, if so - use slave as assigned from spi_init()
- otherwise, use slave as assigned to spi->dev.slave
This allows the AD9361 initialization sequence to go through without
needing to have the spi device populated, and assign it to the appropriate
ad9361_phy once it has been initialized. It appears to have worked for me
since I can now read the two different temperatures of my devices, and
ping-pong the frequencies back with other devices.
I don't know if this is the best solution so I am not posting patches, but
for anyone who is working with multiple AD9361 devices on the same
platform, this will definitely bite you.
Brian
Awesome digging. Thanks for the effort.
On 3/26/20 5:16 PM, Brian Padalino wrote:
On Mon, Feb 24, 2020 at 11:30 AM Brian Padalino bpadalino@gmail.com wrote:
On Mon, Feb 24, 2020 at 11:20 AM Davis Hoover dhoover@geontech.com
wrote:I'm guessing this is an issue with how the framework maps the HDL
slave's memory as made available to the RCC proxy. I know there are
some differences in the C++ classes for management of simulator vs
FPGAs, and it seems I need to exercise the latter to recreate the
problem.It may be helpful to look at your application XML.
I've finally figured out and solved this issue, and it had nothing to do
with the mapping in the HDL. It was a very silly RCC bug that took me ages
to find.
The problem is very straight forward and involves a few files:
- The ad9361_config_proxy.cc calls spi_init() on line 849 which is in
ad9361_platform.cc
- spi_init() assigns the pointer to an instance variable
of OCPI::RCC::RCCUserSlave* called slave
- slave is used whenever spi_write_then_read() is used, and the struct
spi_device* is not used
- Someone planned to use it, since struct spi_device has a device type
which has a void* called slave in it
For my case, each time the initialization of the ad9361_config_proxy was
happening (twice - once for each AFE), spi_init() was being called and kept
re-assigning the slave pointer, even though each ad9361_config_proxy is
separate and have different slaves.
My modification was to assign ad9361_rf_phy->spi->dev.slave after this line:
... and modify the spi_write_then_read() function to:
- check if spi is null, if so - use slave as assigned from spi_init()
- otherwise, use slave as assigned to spi->dev.slave
This allows the AD9361 initialization sequence to go through without
needing to have the spi device populated, and assign it to the appropriate
ad9361_phy once it has been initialized. It appears to have worked for me
since I can now read the two different temperatures of my devices, and
ping-pong the frequencies back with other devices.
I don't know if this is the best solution so I am not posting patches, but
for anyone who is working with multiple AD9361 devices on the same
platform, this will definitely bite you.
Brian
Just wondering if this should be addressed in the 1.7.0 release?
If not, is there an issue I can track?
Thanks,
Brian
On Thu, Mar 26, 2020 at 5:16 PM Brian Padalino bpadalino@gmail.com wrote:
On Mon, Feb 24, 2020 at 11:30 AM Brian Padalino bpadalino@gmail.com
wrote:
On Mon, Feb 24, 2020 at 11:20 AM Davis Hoover dhoover@geontech.com
wrote:
I'm guessing this is an issue with how the framework maps the HDL
slave's memory as made available to the RCC proxy. I know there are
some differences in the C++ classes for management of simulator vs
FPGAs, and it seems I need to exercise the latter to recreate the
problem.
It may be helpful to look at your application XML.
I've finally figured out and solved this issue, and it had nothing to do
with the mapping in the HDL. It was a very silly RCC bug that took me ages
to find.
The problem is very straight forward and involves a few files:
- The ad9361_config_proxy.cc calls spi_init() on line 849 which is in
ad9361_platform.cc - spi_init() assigns the pointer to an instance variable
of OCPI::RCC::RCCUserSlave* called slave - slave is used whenever spi_write_then_read() is used, and the struct
spi_device* is not used - Someone planned to use it, since struct spi_device has a device type
which has a void* called slave in it
For my case, each time the initialization of the ad9361_config_proxy was
happening (twice - once for each AFE), spi_init() was being called and kept
re-assigning the slave pointer, even though each ad9361_config_proxy is
separate and have different slaves.
My modification was to assign ad9361_rf_phy->spi->dev.slave after this
line:
... and modify the spi_write_then_read() function to:
- check if spi is null, if so - use slave as assigned from spi_init()
- otherwise, use slave as assigned to spi->dev.slave
This allows the AD9361 initialization sequence to go through without
needing to have the spi device populated, and assign it to the appropriate
ad9361_phy once it has been initialized. It appears to have worked for me
since I can now read the two different temperatures of my devices, and
ping-pong the frequencies back with other devices.
I don't know if this is the best solution so I am not posting patches, but
for anyone who is working with multiple AD9361 devices on the same
platform, this will definitely bite you.
Brian