Zynq AXI Peripherals in Platform
I have some AXI peripherals that I want to use on the PL side of things
along with OpenCPI. I want to use these in the form of a device tree
overlay to load once the FPGA image has been loaded.
Since OpenCPI loads the FPGA image itself every time it runs, it seems like
OpenCPI needs to be able to perform this operation, right?
Is there a documented method on setting up custom subsystems on the PL side
of things alongside OpenCPI?
Thanks,
Brian
On 3/2/21 10:04 AM, Brian Padalino wrote:
I have some AXI peripherals that I want to use on the PL side of things
along with OpenCPI. I want to use these in the form of a device tree
overlay to load once the FPGA image has been loaded.
Are these AXI slaves, like the OpenCPI control plane?
One way to do this is to use the other AXI slave port (AXI_GP1).
That keeps everything separate.
Then attach your "other" logic to that AXI PS->PL port, in the platform
worker.
But that has nothing to do with device trees and overlays - which can
get into some pretty complicated scenarios, depending on how much you
want to deal with dynamic loading and probing of other linux device
drivers. Not to mention that the whole DTO system only works on pretty
recent Linux kernels.
Are you thinking of partial reconfiguration in this case?
Since OpenCPI loads the FPGA image itself every time it runs, it seems like
OpenCPI needs to be able to perform this operation, right?
Is there a documented method on setting up custom subsystems on the PL side
of things alongside OpenCPI?
Thanks,
Brian
On Tue, Mar 2, 2021 at 10:17 AM James Kulp jek@parera.com wrote:
On 3/2/21 10:04 AM, Brian Padalino wrote:
I have some AXI peripherals that I want to use on the PL side of things
along with OpenCPI. I want to use these in the form of a device tree
overlay to load once the FPGA image has been loaded.
Are these AXI slaves, like the OpenCPI control plane?
These are slaves that are used in the platform by a daemon that control
other aspects of my system. I don't want to re-write the drivers/daemon or
maintain two separate systems. They utilize sysfs entries for opening and
interacting with the devices. I figured it is easier to make a daemon
interface in OpenCPI than bring all the PL IP into it.
One way to do this is to use the other AXI slave port (AXI_GP1).
That keeps everything separate.
Then attach your "other" logic to that AXI PS->PL port, in the platform
worker.
Is there an example of this somewhere in the OpenCPI code?
But that has nothing to do with device trees and overlays - which can
get into some pretty complicated scenarios, depending on how much you
want to deal with dynamic loading and probing of other linux device
drivers. Not to mention that the whole DTO system only works on pretty
recent Linux kernels.
Understandable about the recent nature of this feature. I am worried about
keeping the drivers instantiated when the FPGA image is being reprogrammed
underneath it all, hence the overlays.
Are you thinking of partial reconfiguration in this case?
No.
Brian
@James Kulpmailto:jek@parera.com Would this be something where he would touch the HdlDrivers code? Just like we did with the gpsd gps Daemon?
From: discuss discuss-bounces@lists.opencpi.org on behalf of Brian Padalino bpadalino@gmail.com
Sent: Tuesday, March 2, 2021 10:44 AM
To: James Kulp jek@parera.com
Cc: discuss@lists.opencpi.org discuss@lists.opencpi.org
Subject: Re: [Discuss OpenCPI] Zynq AXI Peripherals in Platform
On Tue, Mar 2, 2021 at 10:17 AM James Kulp jek@parera.com wrote:
On 3/2/21 10:04 AM, Brian Padalino wrote:
I have some AXI peripherals that I want to use on the PL side of things
along with OpenCPI. I want to use these in the form of a device tree
overlay to load once the FPGA image has been loaded.
Are these AXI slaves, like the OpenCPI control plane?
These are slaves that are used in the platform by a daemon that control
other aspects of my system. I don't want to re-write the drivers/daemon or
maintain two separate systems. They utilize sysfs entries for opening and
interacting with the devices. I figured it is easier to make a daemon
interface in OpenCPI than bring all the PL IP into it.
One way to do this is to use the other AXI slave port (AXI_GP1).
That keeps everything separate.
Then attach your "other" logic to that AXI PS->PL port, in the platform
worker.
Is there an example of this somewhere in the OpenCPI code?
But that has nothing to do with device trees and overlays - which can
get into some pretty complicated scenarios, depending on how much you
want to deal with dynamic loading and probing of other linux device
drivers. Not to mention that the whole DTO system only works on pretty
recent Linux kernels.
Understandable about the recent nature of this feature. I am worried about
keeping the drivers instantiated when the FPGA image is being reprogrammed
underneath it all, hence the overlays.
Are you thinking of partial reconfiguration in this case?
No.
Brian
An HTML attachment was scrubbed...
URL: http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20210302/d0f1b8fc/attachment.html
On Tue, Mar 2, 2021 at 2:18 PM James Kulp jek@parera.com wrote:
On 3/2/21 10:44 AM, Brian Padalino wrote:
On Tue, Mar 2, 2021 at 10:17 AM James Kulp jek@parera.com wrote:
On 3/2/21 10:04 AM, Brian Padalino wrote:
I have some AXI peripherals that I want to use on the PL side of things
along with OpenCPI. I want to use these in the form of a device tree
overlay to load once the FPGA image has been loaded.
Are these AXI slaves, like the OpenCPI control plane?
These are slaves that are used in the platform by a daemon that control
other aspects of my system. I don't want to re-write the drivers/daemon or
maintain two separate systems. They utilize sysfs entries for opening and
interacting with the devices. I figured it is easier to make a daemon
interface in OpenCPI than bring all the PL IP into it.
So you can either make a worker that touches sysfs etc., or just a library
if you are using the ACI.
Making a worker is nicer since it enables ocpirun etc.
One way to do this is to use the other AXI slave port (AXI_GP1).
That keeps everything separate.
Then attach your "other" logic to that AXI PS->PL port, in the platform
worker.
Is there an example of this somewhere in the OpenCPI code?
Unfortunately the example we have is proprietary, but basically it is a
module in the platform worker that is attached directly to the GP1
interface from the zynq_ps module in the platform worker.
So this is a use case that OpenCPI wants to support, but the example that
shows this use case isn't open for the OpenCPI project?
Are you able to remove any proprietary aspect and post the example to the
OpenCPI project? Preferably it would be a BD instantiation with multiple
devices using the same AXI interface and the Xilinx Smart Interconnect?
I.e.:
ps_m_axi_gp_in(1).ar.ready <= mymodule.arready;...
mymodule.araddr <= ps_m_axi_gp_out(1).ar.addr
This assumes you can make your other drivers/devices accessed at GP1.
It is also possible to run opencpi using GP1 and leave GP0 for the other
stuff.
The zed platform has this GP0 vs. GP1 option.
But that has nothing to do with device trees and overlays - which can
get into some pretty complicated scenarios, depending on how much you
want to deal with dynamic loading and probing of other linux device
drivers. Not to mention that the whole DTO system only works on pretty
recent Linux kernels.
Understandable about the recent nature of this feature. I am worried
about keeping the drivers instantiated when the FPGA image is being
reprogrammed underneath it all, hence the overlays.
Yes, that is something to worry about. The other drivers would need to be
told something in order to "reinitialize the hardware". Perhaps you could
do that in your "worker".
This doesn't necessarily work since the daemon typically handles loading
the FPGA and overlay.
I guess let me ask the question differently - when will OpenCPI support
device tree overlays and loading FPGA images via that mechanism?
Thanks,
Brian
An HTML attachment was scrubbed...
URL: http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20210302/50221ba2/attachment.html
On 3/2/21 7:47 PM, Brian Padalino wrote:
On Tue, Mar 2, 2021 at 2:18 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:
On 3/2/21 10:44 AM, Brian Padalino wrote:
On Tue, Mar 2, 2021 at 10:17 AM James Kulp <jek@parera.com
<mailto:jek@parera.com>> wrote:
On 3/2/21 10:04 AM, Brian Padalino wrote:
I have some AXI peripherals that I want to use on the PL
side of things
along with OpenCPI. I want to use these in the form of a
device tree
overlay to load once the FPGA image has been loaded.
Are these AXI slaves, like the OpenCPI control plane?
These are slaves that are used in the platform by a daemon that
control other aspects of my system. I don't want to re-write the
drivers/daemon or maintain two separate systems. They utilize
sysfs entries for opening and interacting with the devices. I
figured it is easier to make a daemon interface in OpenCPI than
bring all the PL IP into it.
So you can either make a worker that touches sysfs etc., or just a
library if you are using the ACI.
Making a worker is nicer since it enables ocpirun etc.
One way to do this is to use the other AXI slave port (AXI_GP1).
That keeps everything separate.
Then attach your "other" logic to that AXI PS->PL port, in
the platform
worker.
Is there an example of this somewhere in the OpenCPI code?
Unfortunately the example we have is proprietary, but basically it
is a module in the platform worker that is attached directly to
the GP1 interface from the zynq_ps module in the platform worker.
So this is a use case that OpenCPI wants to support, but the example
that shows this use case isn't open for the OpenCPI project?
It was done once, and the outcome was not very satisfactory since it
degraded the portabilty of the OpenCPI applications.
Are you able to remove any proprietary aspect and post the example to
the OpenCPI project? Preferably it would be a BD instantiation with
multiple devices using the same AXI interface and the Xilinx Smart
Interconnect?
We don't usually use vendor construction tools, but we can probably post
the example after cleaning it up a bit.
I.e.:
ps_m_axi_gp_in(1).ar.ready <= mymodule.arready;...
mymodule.araddr <= ps_m_axi_gp_out(1).ar.addr
This assumes you can make your other drivers/devices accessed at GP1.
It is also possible to run opencpi using GP1 and leave GP0 for the
other stuff.
The zed platform has this GP0 vs. GP1 option.
But that has nothing to do with device trees and overlays -
which can
get into some pretty complicated scenarios, depending on how
much you
want to deal with dynamic loading and probing of other linux
device
drivers. Not to mention that the whole DTO system only
works on pretty
recent Linux kernels.
Understandable about the recent nature of this feature. I am
worried about keeping the drivers instantiated when the FPGA
image is being reprogrammed underneath it all, hence the overlays.
Yes, that is something to worry about. The other drivers would
need to be told something in order to "reinitialize the
hardware". Perhaps you could do that in your "worker".
This doesn't necessarily work since the daemon typically handles
loading the FPGA and overlay.
I guess let me ask the question differently - when will OpenCPI
support device tree overlays and loading FPGA images via that mechanism?
At least at the moment, it is not on any priority roadmap. We already
support about 4 different bitstream loading mechanisms, from 2013 to
2019, all of which are simpler and have more commonalty than the DTO
mechanism, which is very different. That is not to say it is a bad
idea, but sharing the FPGA with arbitrary other code and drivers is
fraught with integration ratholes, and is hard to imagine a "common"
solution, without lots of effort with several use-cases. Using DTO also
folds back into the build system. Up to now all the different
mechanisms have not impacted the build flow, whereas DTO could fork the
build process too, which expands the test matrix. I'm sure it will
happen at some point, but not likely until someone commits to a
significant effort when there is no simpler alternative.
Jim
Thanks,
Brian