I have an application where multiple items need to be read out at the same
time for a single property. They are intimately related to each other. So
I have a structure property with multiple members. I have a readsync setup
so that I can set all the members at once based on some internal states.
But it's very important that they all be updated at the same time.
Multiple reads are not useful.
I saw section 7.1.7.1 of the Application Development guide:
https://opencpi.github.io/OpenCPI_Application_Development.pdf
But it's unclear how to get all the values of the struct out at once. Is
this possible? Am I just reading that document incorrectly?
Thanks,
Brian
The ACI has support for retrieving values for single members of a struct
property in their primitive type, but not all members in their primitive
type. Only all members as a string representation of the entire property.
The string can then be manually parsed. One example of such usage is here:
https://gitlab.com/opencpi/opencpi/-/blob/develop/projects/assets/hdl/cards/fmcomms_2_3_rx.rcc/include/worker_prop_parsers_ad9361_config_proxy.h#L144
From your description it sounds like your property is volatile, for which
your concern of consecutive reads is valid. If the property is not volatile
or readable, then this concern would not entirely be valid, but would
certainly have other considerations.
---------- Forwarded message ---------
From: Brian Padalino bpadalino@gmail.com
Date: Fri, Apr 24, 2020 at 5:54 PM
Subject: [Discuss OpenCPI] Accessing Complex Properties (struct) of A Worker
To: discuss@lists.opencpi.org
I have an application where multiple items need to be read out at the same
time for a single property. They are intimately related to each other. So
I have a structure property with multiple members. I have a readsync setup
so that I can set all the members at once based on some internal states.
But it's very important that they all be updated at the same time.
Multiple reads are not useful.
I saw section 7.1.7.1 of the Application Development guide:
https://opencpi.github.io/OpenCPI_Application_Development.pdf
But it's unclear how to get all the values of the struct out at once. Is
this possible? Am I just reading that document incorrectly?
Thanks,
Brian
I'll put forth a more restricted answer. The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.
These restrictions come down to the fact that OpenCPI does not perform
data type code generation for the ACI.
One idea that has been floated is to use Google's flatbuffers for this
purpose.
Jim
On 4/24/20 8:41 PM, Davis Hoover wrote:
The ACI has support for retrieving values for single members of a struct
property in their primitive type, but not all members in their primitive
type. Only all members as a string representation of the entire property.
The string can then be manually parsed. One example of such usage is here:
https://gitlab.com/opencpi/opencpi/-/blob/develop/projects/assets/hdl/cards/fmcomms_2_3_rx.rcc/include/worker_prop_parsers_ad9361_config_proxy.h#L144
From your description it sounds like your property is volatile, for which
your concern of consecutive reads is valid. If the property is not volatile
or readable, then this concern would not entirely be valid, but would
certainly have other considerations.
---------- Forwarded message ---------
From: Brian Padalino bpadalino@gmail.com
Date: Fri, Apr 24, 2020 at 5:54 PM
Subject: [Discuss OpenCPI] Accessing Complex Properties (struct) of A Worker
To: discuss@lists.opencpi.org
I have an application where multiple items need to be read out at the same
time for a single property. They are intimately related to each other. So
I have a structure property with multiple members. I have a readsync setup
so that I can set all the members at once based on some internal states.
But it's very important that they all be updated at the same time.
Multiple reads are not useful.
I saw section 7.1.7.1 of the Application Development guide:
https://opencpi.github.io/OpenCPI_Application_Development.pdf
But it's unclear how to get all the values of the struct out at once. Is
this possible? Am I just reading that document incorrectly?
Thanks,
Brian
On Sat, Apr 25, 2020 at 8:13 AM James Kulp jek@parera.com wrote:
I'll put forth a more restricted answer. The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.
These restrictions come down to the fact that OpenCPI does not perform
data type code generation for the ACI.
Sure - that's very understandable.
One idea that has been floated is to use Google's flatbuffers for this
purpose.
I won't comment on the binary reading part, but I would definitely
recommend changing the string data exchange format to be JSON instead of
what you have. There is a great JSON library for C++11 which makes super
easy to use objects and is MIT licensed:
https://github.com/nlohmann/json
Now, for something back on topic. I had setup the readsync functionality,
but forgot to set it to volatile. When I did this, the read() method was
called for syncing, but the data was never updated. Is this expected
behavior? I would think that readsync would imply volatile. Are there
cases where readsync would be set, but volatile shouldn't be assumed?
Lastly, there's a lot of XML happening for all these different devices,
workers, platforms, etc. Is there a comprehensive XML schema that can be
referenced for each of the XML data types?
Thanks,
Brian
An HTML attachment was scrubbed...
URL: http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20200427/f49d4ad4/attachment.html
On 4/27/20 10:03 AM, Brian Padalino wrote:
On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:
I'll put forth a more restricted answer. The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.
These restrictions come down to the fact that OpenCPI does not
perform
data type code generation for the ACI.
Sure - that's very understandable.
One idea that has been floated is to use Google's flatbuffers for
this
purpose.
I won't comment on the binary reading part, but I would definitely
recommend changing the string data exchange format to be JSON instead
of what you have. There is a great JSON library for C++11 which makes
super easy to use objects and is MIT licensed:
And then all the yaml devotees will start complaining! IWBNI there was
a single textual format for everything we have. Currently our repo has
4....(XML, YAML, JSON, property-value).)-:
Now, for something back on topic. I had setup the readsync
functionality, but forgot to set it to volatile. When I did this, the
read() method was called for syncing, but the data was never updated.
Is this expected behavior? I would think that readsync would imply
volatile. Are there cases where readsync would be set, but volatile
shouldn't be assumed?
It is expected since you never said the value would change.
Having a readsync imply volatile is sensible, but a bit magical.
It would also eliminate the functionality of a readsync on non-volatile
values (i.e. tell me when it is accessed even if I won't change it after
start).
I.e. it would make the two issues tied together rather than independent
of each other.
Lastly, there's a lot of XML happening for all these different
devices, workers, platforms, etc. Is there a comprehensive XML schema
that can be referenced for each of the XML data types?
Since we use a very compact runtime parser for XML, it was not ever
attractive to have that parser be schema-driven, since that would bloat
it, by 2 orders of magnitude last time we checked.
Even if there was a schema, it is far from sufficient for knowing
whether XML is correct, so there is always more validation that must be
coded separate from the schema.
And schema-based parsers cannot provide semantic errors.
That being said there are three pretty clear deficiencies in this area:
Some people want it to all be yaml, and some want it all to be json :-)
There is no simple standalone validation tool in OpenCPI - you have
to "build" and get errors.
The syntax-level (schema-level) error reporting is not great.
Thanks,
Brian
On Mon, Apr 27, 2020 at 3:48 PM James Kulp jek@parera.com wrote:
On 4/27/20 10:03 AM, Brian Padalino wrote:
On Sat, Apr 25, 2020 at 8:13 AM James Kulp jek@parera.com wrote:
I'll put forth a more restricted answer. The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.
These restrictions come down to the fact that OpenCPI does not perform
data type code generation for the ACI.
Sure - that's very understandable.
One idea that has been floated is to use Google's flatbuffers for this
purpose.
I won't comment on the binary reading part, but I would definitely
recommend changing the string data exchange format to be JSON instead of
what you have. There is a great JSON library for C++11 which makes super
easy to use objects and is MIT licensed:
https://github.com/nlohmann/json
And then all the yaml devotees will start complaining! IWBNI there was a
single textual format for everything we have. Currently our repo has
4....(XML, YAML, JSON, property-value).)-:
I think my suggestion here was lost. I am not suggesting JSON be anywhere
else, but a getPropertyJSON() method would be nice. Right now it spits out
some formatted string which I have to write some parse method for as
described earlier by Davis. YAML doesn't make sense here, but JSON really
does.
Now, for something back on topic. I had setup the readsync functionality,
but forgot to set it to volatile. When I did this, the read() method was
called for syncing, but the data was never updated. Is this expected
behavior? I would think that readsync would imply volatile. Are there
cases where readsync would be set, but volatile shouldn't be assumed?
It is expected since you never said the value would change.
Having a readsync imply volatile is sensible, but a bit magical.
It would also eliminate the functionality of a readsync on non-volatile
values (i.e. tell me when it is accessed even if I won't change it after
start).
I.e. it would make the two issues tied together rather than independent of
each other.
OK. Understandable.
Lastly, there's a lot of XML happening for all these different devices,
workers, platforms, etc. Is there a comprehensive XML schema that can be
referenced for each of the XML data types?
Since we use a very compact runtime parser for XML, it was not ever
attractive to have that parser be schema-driven, since that would bloat it,
by 2 orders of magnitude last time we checked.
Even if there was a schema, it is far from sufficient for knowing whether
XML is correct, so there is always more validation that must be coded
separate from the schema.
And schema-based parsers cannot provide semantic errors.
That being said there are three pretty clear deficiencies in this area:
Some people want it to all be yaml, and some want it all to be json :-)
There is no simple standalone validation tool in OpenCPI - you have to
"build" and get errors.
The syntax-level (schema-level) error reporting is not great.
OK. How about a different way of asking the question.
The XML files are defined across a whole slew of documentation in many
different PDF's and not coherently.
Is there a centralized document which describes each XML entity with their
properties and what they all mean? Sometimes I get errors stating what was
expected when I don't type something that is expected. That's all I am
asking for. I don't care about validation. I just want to know what's
possible without having to look mostly at examples or 5 different PDFs.
Thanks,
Brian
On 4/27/20 4:05 PM, Brian Padalino wrote:
On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:
On 4/27/20 10:03 AM, Brian Padalino wrote:
On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com
<mailto:jek@parera.com>> wrote:
I'll put forth a more restricted answer. The ACI can read
volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted
string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly
as binary
values.
These restrictions come down to the fact that OpenCPI does
not perform
data type code generation for the ACI.
Sure - that's very understandable.
One idea that has been floated is to use Google's flatbuffers
for this
purpose.
I won't comment on the binary reading part, but I would
definitely recommend changing the string data exchange format to
be JSON instead of what you have. There is a great JSON library
for C++11 which makes super easy to use objects and is MIT licensed:
https://github.com/nlohmann/json
And then all the yaml devotees will start complaining! IWBNI there
was a single textual format for everything we have. Currently our
repo has 4....(XML, YAML, JSON, property-value).)-:
I think my suggestion here was lost. I am not suggesting JSON be
anywhere else, but a getPropertyJSON() method would be nice. Right
now it spits out some formatted string which I have to write some
parse method for as described earlier by Davis. YAML doesn't make
sense here, but JSON really does.
Now, for something back on topic. I had setup the readsync
functionality, but forgot to set it to volatile. When I did
this, the read() method was called for syncing, but the data was
never updated. Is this expected behavior? I would think that
readsync would imply volatile. Are there cases where readsync
would be set, but volatile shouldn't be assumed?
It is expected since you never said the value would change.
Having a readsync imply volatile is sensible, but a bit magical.
It would also eliminate the functionality of a readsync on
non-volatile values (i.e. tell me when it is accessed even if I
won't change it after start).
I.e. it would make the two issues tied together rather than
independent of each other.
OK. Understandable.
Lastly, there's a lot of XML happening for all these different
devices, workers, platforms, etc. Is there a comprehensive XML
schema that can be referenced for each of the XML data types?
Since we use a very compact runtime parser for XML, it was not
ever attractive to have that parser be schema-driven, since that
would bloat it, by 2 orders of magnitude last time we checked.
Even if there was a schema, it is far from sufficient for knowing
whether XML is correct, so there is always more validation that
must be coded separate from the schema.
And schema-based parsers cannot provide semantic errors.
That being said there are three pretty clear deficiencies in this
area:
1. Some people want it to all be yaml, and some want it all to be
json :-)
2. There is no simple standalone validation tool in OpenCPI - you
have to "build" and get errors.
3. The syntax-level (schema-level) error reporting is not great.
OK. How about a different way of asking the question.
The XML files are defined across a whole slew of documentation in many
different PDF's and not coherently.
Is there a centralized document which describes each XML entity with
their properties and what they all mean? Sometimes I get errors
stating what was expected when I don't type something that is
expected. That's all I am asking for. I don't care about
validation. I just want to know what's possible without having to
look mostly at examples or 5 different PDFs.
Sorry to make it overly complicated. There is
doc/xls/OpenCPI_XML_Attributes.xlsx
It is not up to date, but if you like it, perhaps it can be.:-)
Thanks,
Brian
JSON is a great suggestion.
On Mon, Apr 27, 2020 at 4:11 PM James Kulp jek@parera.com wrote:
On 4/27/20 4:05 PM, Brian Padalino wrote:
On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:
On 4/27/20 10:03 AM, Brian Padalino wrote:
On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com
<mailto:jek@parera.com>> wrote:
I'll put forth a more restricted answer. The ACI can read
volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted
string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly
as binary
values.
These restrictions come down to the fact that OpenCPI does
not perform
data type code generation for the ACI.
Sure - that's very understandable.
One idea that has been floated is to use Google's flatbuffers
for this
purpose.
I won't comment on the binary reading part, but I would
definitely recommend changing the string data exchange format to
be JSON instead of what you have. There is a great JSON library
for C++11 which makes super easy to use objects and is MIT licensed:
https://github.com/nlohmann/json
And then all the yaml devotees will start complaining! IWBNI there
was a single textual format for everything we have. Currently our
repo has 4....(XML, YAML, JSON, property-value).)-:
I think my suggestion here was lost. I am not suggesting JSON be
anywhere else, but a getPropertyJSON() method would be nice. Right
now it spits out some formatted string which I have to write some
parse method for as described earlier by Davis. YAML doesn't make
sense here, but JSON really does.
Now, for something back on topic. I had setup the readsync
functionality, but forgot to set it to volatile. When I did
this, the read() method was called for syncing, but the data was
never updated. Is this expected behavior? I would think that
readsync would imply volatile. Are there cases where readsync
would be set, but volatile shouldn't be assumed?
It is expected since you never said the value would change.
Having a readsync imply volatile is sensible, but a bit magical.
It would also eliminate the functionality of a readsync on
non-volatile values (i.e. tell me when it is accessed even if I
won't change it after start).
I.e. it would make the two issues tied together rather than
independent of each other.
OK. Understandable.
Lastly, there's a lot of XML happening for all these different
devices, workers, platforms, etc. Is there a comprehensive XML
schema that can be referenced for each of the XML data types?
Since we use a very compact runtime parser for XML, it was not
ever attractive to have that parser be schema-driven, since that
would bloat it, by 2 orders of magnitude last time we checked.
Even if there was a schema, it is far from sufficient for knowing
whether XML is correct, so there is always more validation that
must be coded separate from the schema.
And schema-based parsers cannot provide semantic errors.
That being said there are three pretty clear deficiencies in this
area:
1. Some people want it to all be yaml, and some want it all to be
json :-)
2. There is no simple standalone validation tool in OpenCPI - you
have to "build" and get errors.
3. The syntax-level (schema-level) error reporting is not great.
OK. How about a different way of asking the question.
The XML files are defined across a whole slew of documentation in many
different PDF's and not coherently.
Is there a centralized document which describes each XML entity with
their properties and what they all mean? Sometimes I get errors
stating what was expected when I don't type something that is
expected. That's all I am asking for. I don't care about
validation. I just want to know what's possible without having to
look mostly at examples or 5 different PDFs.
Sorry to make it overly complicated. There is
doc/xls/OpenCPI_XML_Attributes.xlsx
It is not up to date, but if you like it, perhaps it can be.:-)
Thanks,
Brian
An HTML attachment was scrubbed...
URL: http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20200429/14041e84/attachment.html