Accessing Complex Properties (struct) of A Worker

BP
Brian Padalino
Fri, Apr 24, 2020 9:53 PM

I have an application where multiple items need to be read out at the same
time for a single property.  They are intimately related to each other.  So
I have a structure property with multiple members.  I have a readsync setup
so that I can set all the members at once based on some internal states.
But it's very important that they all be updated at the same time.
Multiple reads are not useful.

I saw section 7.1.7.1 of the Application Development guide:

https://opencpi.github.io/OpenCPI_Application_Development.pdf

But it's unclear how to get all the values of the struct out at once.  Is
this possible?  Am I just reading that document incorrectly?

Thanks,
Brian

I have an application where multiple items need to be read out at the same time for a single property. They are intimately related to each other. So I have a structure property with multiple members. I have a readsync setup so that I can set all the members at once based on some internal states. But it's very important that they all be updated at the same time. Multiple reads are not useful. I saw section 7.1.7.1 of the Application Development guide: https://opencpi.github.io/OpenCPI_Application_Development.pdf But it's unclear how to get all the values of the struct out at once. Is this possible? Am I just reading that document incorrectly? Thanks, Brian
DH
Davis Hoover
Sat, Apr 25, 2020 12:41 AM

The ACI has support for retrieving values for single members of a struct
property in their primitive type, but not all members in their primitive
type. Only all members as a string representation of the entire property.
The string can then be manually parsed. One example of such usage is here:
https://gitlab.com/opencpi/opencpi/-/blob/develop/projects/assets/hdl/cards/fmcomms_2_3_rx.rcc/include/worker_prop_parsers_ad9361_config_proxy.h#L144

From your description it sounds like your property is volatile, for which
your concern of consecutive reads is valid. If the property is not volatile
or readable, then this concern would not entirely be valid, but would
certainly have other considerations.

---------- Forwarded message ---------
From: Brian Padalino bpadalino@gmail.com
Date: Fri, Apr 24, 2020 at 5:54 PM
Subject: [Discuss OpenCPI] Accessing Complex Properties (struct) of A Worker
To: discuss@lists.opencpi.org

I have an application where multiple items need to be read out at the same
time for a single property.  They are intimately related to each other.  So
I have a structure property with multiple members.  I have a readsync setup
so that I can set all the members at once based on some internal states.
But it's very important that they all be updated at the same time.
Multiple reads are not useful.

I saw section 7.1.7.1 of the Application Development guide:

https://opencpi.github.io/OpenCPI_Application_Development.pdf

But it's unclear how to get all the values of the struct out at once.  Is
this possible?  Am I just reading that document incorrectly?

Thanks,
Brian

The ACI has support for retrieving values for *single members* of a struct property in their primitive type, but not *all* members in their primitive type. Only all members as a string representation of the entire property. The string can then be manually parsed. One example of such usage is here: https://gitlab.com/opencpi/opencpi/-/blob/develop/projects/assets/hdl/cards/fmcomms_2_3_rx.rcc/include/worker_prop_parsers_ad9361_config_proxy.h#L144 From your description it sounds like your property is volatile, for which your concern of consecutive reads is valid. If the property is not volatile or readable, then this concern would not entirely be valid, but would certainly have other considerations. ---------- Forwarded message --------- From: Brian Padalino <bpadalino@gmail.com> Date: Fri, Apr 24, 2020 at 5:54 PM Subject: [Discuss OpenCPI] Accessing Complex Properties (struct) of A Worker To: <discuss@lists.opencpi.org> I have an application where multiple items need to be read out at the same time for a single property. They are intimately related to each other. So I have a structure property with multiple members. I have a readsync setup so that I can set all the members at once based on some internal states. But it's very important that they all be updated at the same time. Multiple reads are not useful. I saw section 7.1.7.1 of the Application Development guide: https://opencpi.github.io/OpenCPI_Application_Development.pdf But it's unclear how to get all the values of the struct out at once. Is this possible? Am I just reading that document incorrectly? Thanks, Brian
JK
James Kulp
Sat, Apr 25, 2020 12:13 PM

I'll put forth a more restricted answer.  The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.

These restrictions come down to the fact that OpenCPI does not perform
data type code generation for the ACI.

One idea that has been floated is to use Google's flatbuffers for this
purpose.

Jim

On 4/24/20 8:41 PM, Davis Hoover wrote:

The ACI has support for retrieving values for single members of a struct
property in their primitive type, but not all members in their primitive
type. Only all members as a string representation of the entire property.
The string can then be manually parsed. One example of such usage is here:
https://gitlab.com/opencpi/opencpi/-/blob/develop/projects/assets/hdl/cards/fmcomms_2_3_rx.rcc/include/worker_prop_parsers_ad9361_config_proxy.h#L144

From your description it sounds like your property is volatile, for which
your concern of consecutive reads is valid. If the property is not volatile
or readable, then this concern would not entirely be valid, but would
certainly have other considerations.

---------- Forwarded message ---------
From: Brian Padalino bpadalino@gmail.com
Date: Fri, Apr 24, 2020 at 5:54 PM
Subject: [Discuss OpenCPI] Accessing Complex Properties (struct) of A Worker
To: discuss@lists.opencpi.org

I have an application where multiple items need to be read out at the same
time for a single property.  They are intimately related to each other.  So
I have a structure property with multiple members.  I have a readsync setup
so that I can set all the members at once based on some internal states.
But it's very important that they all be updated at the same time.
Multiple reads are not useful.

I saw section 7.1.7.1 of the Application Development guide:

https://opencpi.github.io/OpenCPI_Application_Development.pdf

But it's unclear how to get all the values of the struct out at once.  Is
this possible?  Am I just reading that document incorrectly?

Thanks,
Brian

I'll put forth a more restricted answer.  The ACI can read volatile property values of all types, whole and in part. In all cases you can get the value returned as a formatted string according to the documented textual format. When the values being read (whole or part) are scalars or arrays/sequences of scalars, they can also be read directly as binary values. These restrictions come down to the fact that OpenCPI does not perform data type code generation for the ACI. One idea that has been floated is to use Google's flatbuffers for this purpose. Jim On 4/24/20 8:41 PM, Davis Hoover wrote: > The ACI has support for retrieving values for *single members* of a struct > property in their primitive type, but not *all* members in their primitive > type. Only all members as a string representation of the entire property. > The string can then be manually parsed. One example of such usage is here: > https://gitlab.com/opencpi/opencpi/-/blob/develop/projects/assets/hdl/cards/fmcomms_2_3_rx.rcc/include/worker_prop_parsers_ad9361_config_proxy.h#L144 > > From your description it sounds like your property is volatile, for which > your concern of consecutive reads is valid. If the property is not volatile > or readable, then this concern would not entirely be valid, but would > certainly have other considerations. > > ---------- Forwarded message --------- > From: Brian Padalino <bpadalino@gmail.com> > Date: Fri, Apr 24, 2020 at 5:54 PM > Subject: [Discuss OpenCPI] Accessing Complex Properties (struct) of A Worker > To: <discuss@lists.opencpi.org> > > > I have an application where multiple items need to be read out at the same > time for a single property. They are intimately related to each other. So > I have a structure property with multiple members. I have a readsync setup > so that I can set all the members at once based on some internal states. > But it's very important that they all be updated at the same time. > Multiple reads are not useful. > > I saw section 7.1.7.1 of the Application Development guide: > > https://opencpi.github.io/OpenCPI_Application_Development.pdf > > But it's unclear how to get all the values of the struct out at once. Is > this possible? Am I just reading that document incorrectly? > > Thanks, > Brian >
BP
Brian Padalino
Mon, Apr 27, 2020 2:03 PM

On Sat, Apr 25, 2020 at 8:13 AM James Kulp jek@parera.com wrote:

I'll put forth a more restricted answer.  The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.

These restrictions come down to the fact that OpenCPI does not perform
data type code generation for the ACI.

Sure - that's very understandable.

One idea that has been floated is to use Google's flatbuffers for this
purpose.

I won't comment on the binary reading part, but I would definitely
recommend changing the string data exchange format to be JSON instead of
what you have.  There is a great JSON library for C++11 which makes super
easy to use objects and is MIT licensed:

https://github.com/nlohmann/json

Now, for something back on topic.  I had setup the readsync functionality,
but forgot to set it to volatile.  When I did this, the read() method was
called for syncing, but the data was never updated.  Is this expected
behavior?  I would think that readsync would imply volatile.  Are there
cases where readsync would be set, but volatile shouldn't be assumed?

Lastly, there's a lot of XML happening for all these different devices,
workers, platforms, etc.  Is there a comprehensive XML schema that can be
referenced for each of the XML data types?

Thanks,
Brian

On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com> wrote: > I'll put forth a more restricted answer. The ACI can read volatile > property values of all types, whole and in part. > In all cases you can get the value returned as a formatted string > according to the documented textual format. > When the values being read (whole or part) are scalars or > arrays/sequences of scalars, they can also be read directly as binary > values. > > These restrictions come down to the fact that OpenCPI does not perform > data type code generation for the ACI. > Sure - that's very understandable. > One idea that has been floated is to use Google's flatbuffers for this > purpose. > I won't comment on the binary reading part, but I would definitely recommend changing the string data exchange format to be JSON instead of what you have. There is a great JSON library for C++11 which makes super easy to use objects and is MIT licensed: https://github.com/nlohmann/json Now, for something back on topic. I had setup the readsync functionality, but forgot to set it to volatile. When I did this, the read() method was called for syncing, but the data was never updated. Is this expected behavior? I would think that readsync would imply volatile. Are there cases where readsync would be set, but volatile shouldn't be assumed? Lastly, there's a lot of XML happening for all these different devices, workers, platforms, etc. Is there a comprehensive XML schema that can be referenced for each of the XML data types? Thanks, Brian
JK
James Kulp
Mon, Apr 27, 2020 7:48 PM
An HTML attachment was scrubbed... URL: <http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20200427/f49d4ad4/attachment.html>
JK
James Kulp
Mon, Apr 27, 2020 8:03 PM

On 4/27/20 10:03 AM, Brian Padalino wrote:

On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:

 I'll put forth a more restricted answer.  The ACI can read volatile
 property values of all types, whole and in part.
 In all cases you can get the value returned as a formatted string
 according to the documented textual format.
 When the values being read (whole or part) are scalars or
 arrays/sequences of scalars, they can also be read directly as binary
 values.

 These restrictions come down to the fact that OpenCPI does not
 perform
 data type code generation for the ACI.

Sure - that's very understandable.

 One idea that has been floated is to use Google's flatbuffers for
 this
 purpose.

I won't comment on the binary reading part, but I would definitely
recommend changing the string data exchange format to be JSON instead
of what you have.  There is a great JSON library for C++11 which makes
super easy to use objects and is MIT licensed:

https://github.com/nlohmann/json

And then all the yaml devotees will start complaining!  IWBNI there was
a single textual format for everything we have.  Currently our repo has
4....(XML, YAML, JSON, property-value).)-:

Now, for something back on topic.  I had setup the readsync
functionality, but forgot to set it to volatile. When I did this, the
read() method was called for syncing, but the data was never updated. 
Is this expected behavior? I would think that readsync would imply
volatile.  Are there cases where readsync would be set, but volatile
shouldn't be assumed?

It is expected since you never said the value would change.

Having a readsync imply volatile is sensible, but a bit magical.

It would also eliminate the functionality of a readsync on non-volatile
values (i.e. tell me when it is accessed even if I won't change it after
start).
I.e. it would make the two issues tied together rather than independent
of each other.

Lastly, there's a lot of XML happening for all these different
devices, workers, platforms, etc.  Is there a comprehensive XML schema
that can be referenced for each of the XML data types?

Since we use a very compact runtime parser for XML, it was not ever
attractive to have that parser be schema-driven, since that would bloat
it, by 2 orders of magnitude last time we checked.

Even if there was a schema, it is far from sufficient for knowing
whether XML is correct, so there is always more validation that must be
coded separate from the schema.

And schema-based parsers cannot provide semantic errors.

That being said there are three pretty clear deficiencies in this area:

  1. Some people want it to all be yaml, and some want it all to be json :-)

  2. There is no simple standalone validation tool in OpenCPI - you have
    to "build" and get errors.

  3. The syntax-level (schema-level) error reporting is not great.

Thanks,
Brian

On 4/27/20 10:03 AM, Brian Padalino wrote: > On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com > <mailto:jek@parera.com>> wrote: > > I'll put forth a more restricted answer.  The ACI can read volatile > property values of all types, whole and in part. > In all cases you can get the value returned as a formatted string > according to the documented textual format. > When the values being read (whole or part) are scalars or > arrays/sequences of scalars, they can also be read directly as binary > values. > > These restrictions come down to the fact that OpenCPI does not > perform > data type code generation for the ACI. > > > Sure - that's very understandable. > > > One idea that has been floated is to use Google's flatbuffers for > this > purpose. > > > I won't comment on the binary reading part, but I would definitely > recommend changing the string data exchange format to be JSON instead > of what you have.  There is a great JSON library for C++11 which makes > super easy to use objects and is MIT licensed: > > https://github.com/nlohmann/json > And then all the yaml devotees will start complaining!  IWBNI there was a single textual format for everything we have.  Currently our repo has 4....(XML, YAML, JSON, property-value).)-: > Now, for something back on topic.  I had setup the readsync > functionality, but forgot to set it to volatile. When I did this, the > read() method was called for syncing, but the data was never updated.  > Is this expected behavior? I would think that readsync would imply > volatile.  Are there cases where readsync would be set, but volatile > shouldn't be assumed? It is expected since you never said the value would change. Having a readsync imply volatile is sensible, but a bit magical. It would also eliminate the functionality of a readsync on non-volatile values (i.e. tell me when it is accessed even if I won't change it after start). I.e. it would make the two issues tied together rather than independent of each other. > > Lastly, there's a lot of XML happening for all these different > devices, workers, platforms, etc.  Is there a comprehensive XML schema > that can be referenced for each of the XML data types? Since we use a very compact runtime parser for XML, it was not ever attractive to have that parser be schema-driven, since that would bloat it, by 2 orders of magnitude last time we checked. Even if there was a schema, it is far from sufficient for knowing whether XML is correct, so there is always more validation that must be coded separate from the schema. And schema-based parsers cannot provide semantic errors. That being said there are three pretty clear deficiencies in this area: 1. Some people want it to all be yaml, and some want it all to be json :-) 2. There is no simple standalone validation tool in OpenCPI - you have to "build" and get errors. 3. The syntax-level (schema-level) error reporting is not great. > > Thanks, > Brian
BP
Brian Padalino
Mon, Apr 27, 2020 8:05 PM

On Mon, Apr 27, 2020 at 3:48 PM James Kulp jek@parera.com wrote:

On 4/27/20 10:03 AM, Brian Padalino wrote:

On Sat, Apr 25, 2020 at 8:13 AM James Kulp jek@parera.com wrote:

I'll put forth a more restricted answer.  The ACI can read volatile
property values of all types, whole and in part.
In all cases you can get the value returned as a formatted string
according to the documented textual format.
When the values being read (whole or part) are scalars or
arrays/sequences of scalars, they can also be read directly as binary
values.

These restrictions come down to the fact that OpenCPI does not perform
data type code generation for the ACI.

Sure - that's very understandable.

One idea that has been floated is to use Google's flatbuffers for this
purpose.

I won't comment on the binary reading part, but I would definitely
recommend changing the string data exchange format to be JSON instead of
what you have.  There is a great JSON library for C++11 which makes super
easy to use objects and is MIT licensed:

https://github.com/nlohmann/json

And then all the yaml devotees will start complaining!  IWBNI there was a
single textual format for everything we have.  Currently our repo has
4....(XML, YAML, JSON, property-value).)-:

I think my suggestion here was lost.  I am not suggesting JSON be anywhere
else, but a getPropertyJSON() method would be nice.  Right now it spits out
some formatted string which I have to write some parse method for as
described earlier by Davis.  YAML doesn't make sense here, but JSON really
does.

Now, for something back on topic.  I had setup the readsync functionality,
but forgot to set it to volatile.  When I did this, the read() method was
called for syncing, but the data was never updated.  Is this expected
behavior?  I would think that readsync would imply volatile.  Are there
cases where readsync would be set, but volatile shouldn't be assumed?

It is expected since you never said the value would change.

Having a readsync imply volatile is sensible, but a bit magical.

It would also eliminate the functionality of a readsync on non-volatile
values (i.e. tell me when it is accessed even if I won't change it after
start).
I.e. it would make the two issues tied together rather than independent of
each other.

OK.  Understandable.

Lastly, there's a lot of XML happening for all these different devices,
workers, platforms, etc.  Is there a comprehensive XML schema that can be
referenced for each of the XML data types?

Since we use a very compact runtime parser for XML, it was not ever
attractive to have that parser be schema-driven, since that would bloat it,
by 2 orders of magnitude last time we checked.

Even if there was a schema, it is far from sufficient for knowing whether
XML is correct, so there is always more validation that must be coded
separate from the schema.

And schema-based parsers cannot provide semantic errors.

That being said there are three pretty clear deficiencies in this area:

  1. Some people want it to all be yaml, and some want it all to be json :-)

  2. There is no simple standalone validation tool in OpenCPI - you have to
    "build" and get errors.

  3. The syntax-level (schema-level) error reporting is not great.

OK.  How about a different way of asking the question.

The XML files are defined across a whole slew of documentation in many
different PDF's and not coherently.

Is there a centralized document which describes each XML entity with their
properties and what they all mean?  Sometimes I get errors stating what was
expected when I don't type something that is expected.  That's all I am
asking for.  I don't care about validation.  I just want to know what's
possible without having to look mostly at examples or 5 different PDFs.

Thanks,
Brian

On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com> wrote: > On 4/27/20 10:03 AM, Brian Padalino wrote: > > On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com> wrote: > >> I'll put forth a more restricted answer. The ACI can read volatile >> property values of all types, whole and in part. >> In all cases you can get the value returned as a formatted string >> according to the documented textual format. >> When the values being read (whole or part) are scalars or >> arrays/sequences of scalars, they can also be read directly as binary >> values. >> >> These restrictions come down to the fact that OpenCPI does not perform >> data type code generation for the ACI. >> > > Sure - that's very understandable. > > >> One idea that has been floated is to use Google's flatbuffers for this >> purpose. >> > > I won't comment on the binary reading part, but I would definitely > recommend changing the string data exchange format to be JSON instead of > what you have. There is a great JSON library for C++11 which makes super > easy to use objects and is MIT licensed: > > https://github.com/nlohmann/json > > And then all the yaml devotees will start complaining! IWBNI there was a > single textual format for everything we have. Currently our repo has > 4....(XML, YAML, JSON, property-value).)-: > I think my suggestion here was lost. I am not suggesting JSON be anywhere else, but a getPropertyJSON() method would be nice. Right now it spits out some formatted string which I have to write some parse method for as described earlier by Davis. YAML doesn't make sense here, but JSON really does. > Now, for something back on topic. I had setup the readsync functionality, > but forgot to set it to volatile. When I did this, the read() method was > called for syncing, but the data was never updated. Is this expected > behavior? I would think that readsync would imply volatile. Are there > cases where readsync would be set, but volatile shouldn't be assumed? > > It is expected since you never said the value would change. > > Having a readsync imply volatile is sensible, but a bit magical. > > It would also eliminate the functionality of a readsync on non-volatile > values (i.e. tell me when it is accessed even if I won't change it after > start). > I.e. it would make the two issues tied together rather than independent of > each other. > OK. Understandable. > > Lastly, there's a lot of XML happening for all these different devices, > workers, platforms, etc. Is there a comprehensive XML schema that can be > referenced for each of the XML data types? > > Since we use a very compact runtime parser for XML, it was not ever > attractive to have that parser be schema-driven, since that would bloat it, > by 2 orders of magnitude last time we checked. > > Even if there was a schema, it is far from sufficient for knowing whether > XML is correct, so there is always more validation that must be coded > separate from the schema. > > And schema-based parsers cannot provide semantic errors. > > That being said there are three pretty clear deficiencies in this area: > 1. Some people want it to all be yaml, and some want it all to be json :-) > > 2. There is no simple standalone validation tool in OpenCPI - you have to > "build" and get errors. > > 3. The syntax-level (schema-level) error reporting is not great. > OK. How about a different way of asking the question. The XML files are defined across a whole slew of documentation in many different PDF's and not coherently. Is there a centralized document which describes each XML entity with their properties and what they all mean? Sometimes I get errors stating what was expected when I don't type something that is expected. That's all I am asking for. I don't care about validation. I just want to know what's possible without having to look mostly at examples or 5 different PDFs. Thanks, Brian
JK
James Kulp
Mon, Apr 27, 2020 8:11 PM

On 4/27/20 4:05 PM, Brian Padalino wrote:

On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:

 On 4/27/20 10:03 AM, Brian Padalino wrote:
 On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com
 <mailto:jek@parera.com>> wrote:

     I'll put forth a more restricted answer.  The ACI can read
     volatile
     property values of all types, whole and in part.
     In all cases you can get the value returned as a formatted
     string
     according to the documented textual format.
     When the values being read (whole or part) are scalars or
     arrays/sequences of scalars, they can also be read directly
     as binary
     values.

     These restrictions come down to the fact that OpenCPI does
     not perform
     data type code generation for the ACI.


 Sure - that's very understandable.


     One idea that has been floated is to use Google's flatbuffers
     for this
     purpose.


 I won't comment on the binary reading part, but I would
 definitely recommend changing the string data exchange format to
 be JSON instead of what you have.  There is a great JSON library
 for C++11 which makes super easy to use objects and is MIT licensed:

 https://github.com/nlohmann/json
 And then all the yaml devotees will start complaining! IWBNI there
 was a single textual format for everything we have.  Currently our
 repo has 4....(XML, YAML, JSON, property-value).)-:

I think my suggestion here was lost.  I am not suggesting JSON be
anywhere else, but a getPropertyJSON() method would be nice.  Right
now it spits out some formatted string which I have to write some
parse method for as described earlier by Davis.  YAML doesn't make
sense here, but JSON really does.

 Now, for something back on topic.  I had setup the readsync
 functionality, but forgot to set it to volatile.  When I did
 this, the read() method was called for syncing, but the data was
 never updated.  Is this expected behavior?  I would think that
 readsync would imply volatile.  Are there cases where readsync
 would be set, but volatile shouldn't be assumed?
 It is expected since you never said the value would change.

 Having a readsync imply volatile is sensible, but a bit magical.

 It would also eliminate the functionality of a readsync on
 non-volatile values (i.e. tell me when it is accessed even if I
 won't change it after start).
 I.e. it would make the two issues tied together rather than
 independent of each other.

 OK.  Understandable.

 Lastly, there's a lot of XML happening for all these different
 devices, workers, platforms, etc. Is there a comprehensive XML
 schema that can be referenced for each of the XML data types?
 Since we use a very compact runtime parser for XML, it was not
 ever attractive to have that parser be schema-driven, since that
 would bloat it, by 2 orders of magnitude last time we checked.

 Even if there was a schema, it is far from sufficient for knowing
 whether XML is correct, so there is always more validation that
 must be coded separate from the schema.

 And schema-based parsers cannot provide semantic errors.

 That being said there are three pretty clear deficiencies in this
 area:
 1. Some people want it to all be yaml, and some want it all to be
 json :-)

 2. There is no simple standalone validation tool in OpenCPI - you
 have to "build" and get errors.

 3. The syntax-level (schema-level) error reporting is not great.

OK.  How about a different way of asking the question.

The XML files are defined across a whole slew of documentation in many
different PDF's and not coherently.

Is there a centralized document which describes each XML entity with
their properties and what they all mean? Sometimes I get errors
stating what was expected when I don't type something that is
expected.  That's all I am asking for.  I don't care about
validation.  I just want to know what's possible without having to
look mostly at examples or 5 different PDFs.

Sorry to make it overly complicated.  There is
doc/xls/OpenCPI_XML_Attributes.xlsx

It is not up to date, but if you like it, perhaps it can be.:-)

Thanks,
Brian

On 4/27/20 4:05 PM, Brian Padalino wrote: > On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com > <mailto:jek@parera.com>> wrote: > > On 4/27/20 10:03 AM, Brian Padalino wrote: >> On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com >> <mailto:jek@parera.com>> wrote: >> >> I'll put forth a more restricted answer.  The ACI can read >> volatile >> property values of all types, whole and in part. >> In all cases you can get the value returned as a formatted >> string >> according to the documented textual format. >> When the values being read (whole or part) are scalars or >> arrays/sequences of scalars, they can also be read directly >> as binary >> values. >> >> These restrictions come down to the fact that OpenCPI does >> not perform >> data type code generation for the ACI. >> >> >> Sure - that's very understandable. >> >> >> One idea that has been floated is to use Google's flatbuffers >> for this >> purpose. >> >> >> I won't comment on the binary reading part, but I would >> definitely recommend changing the string data exchange format to >> be JSON instead of what you have.  There is a great JSON library >> for C++11 which makes super easy to use objects and is MIT licensed: >> >> https://github.com/nlohmann/json >> > And then all the yaml devotees will start complaining! IWBNI there > was a single textual format for everything we have.  Currently our > repo has 4....(XML, YAML, JSON, property-value).)-: > > > I think my suggestion here was lost.  I am not suggesting JSON be > anywhere else, but a getPropertyJSON() method would be nice.  Right > now it spits out some formatted string which I have to write some > parse method for as described earlier by Davis.  YAML doesn't make > sense here, but JSON really does. > >> Now, for something back on topic.  I had setup the readsync >> functionality, but forgot to set it to volatile.  When I did >> this, the read() method was called for syncing, but the data was >> never updated.  Is this expected behavior?  I would think that >> readsync would imply volatile.  Are there cases where readsync >> would be set, but volatile shouldn't be assumed? > > It is expected since you never said the value would change. > > Having a readsync imply volatile is sensible, but a bit magical. > > It would also eliminate the functionality of a readsync on > non-volatile values (i.e. tell me when it is accessed even if I > won't change it after start). > I.e. it would make the two issues tied together rather than > independent of each other. > >  OK.  Understandable. > >> >> Lastly, there's a lot of XML happening for all these different >> devices, workers, platforms, etc. Is there a comprehensive XML >> schema that can be referenced for each of the XML data types? > Since we use a very compact runtime parser for XML, it was not > ever attractive to have that parser be schema-driven, since that > would bloat it, by 2 orders of magnitude last time we checked. > > Even if there was a schema, it is far from sufficient for knowing > whether XML is correct, so there is always more validation that > must be coded separate from the schema. > > And schema-based parsers cannot provide semantic errors. > > That being said there are three pretty clear deficiencies in this > area: > 1. Some people want it to all be yaml, and some want it all to be > json :-) > > 2. There is no simple standalone validation tool in OpenCPI - you > have to "build" and get errors. > > 3. The syntax-level (schema-level) error reporting is not great. > > OK.  How about a different way of asking the question. > > The XML files are defined across a whole slew of documentation in many > different PDF's and not coherently. > > Is there a centralized document which describes each XML entity with > their properties and what they all mean? Sometimes I get errors > stating what was expected when I don't type something that is > expected.  That's all I am asking for.  I don't care about > validation.  I just want to know what's possible without having to > look mostly at examples or 5 different PDFs. Sorry to make it overly complicated.  There is doc/xls/OpenCPI_XML_Attributes.xlsx It is not up to date, but if you like it, perhaps it can be.:-) > > Thanks, > Brian
CH
Chris Hinkey
Mon, Apr 27, 2020 8:48 PM

JSON is a great suggestion.

On Mon, Apr 27, 2020 at 4:11 PM James Kulp jek@parera.com wrote:

On 4/27/20 4:05 PM, Brian Padalino wrote:

On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:

 On 4/27/20 10:03 AM, Brian Padalino wrote:
 On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com
 <mailto:jek@parera.com>> wrote:

     I'll put forth a more restricted answer.  The ACI can read
     volatile
     property values of all types, whole and in part.
     In all cases you can get the value returned as a formatted
     string
     according to the documented textual format.
     When the values being read (whole or part) are scalars or
     arrays/sequences of scalars, they can also be read directly
     as binary
     values.

     These restrictions come down to the fact that OpenCPI does
     not perform
     data type code generation for the ACI.


 Sure - that's very understandable.


     One idea that has been floated is to use Google's flatbuffers
     for this
     purpose.


 I won't comment on the binary reading part, but I would
 definitely recommend changing the string data exchange format to
 be JSON instead of what you have.  There is a great JSON library
 for C++11 which makes super easy to use objects and is MIT licensed:

 https://github.com/nlohmann/json
 And then all the yaml devotees will start complaining! IWBNI there
 was a single textual format for everything we have.  Currently our
 repo has 4....(XML, YAML, JSON, property-value).)-:

I think my suggestion here was lost.  I am not suggesting JSON be
anywhere else, but a getPropertyJSON() method would be nice.  Right
now it spits out some formatted string which I have to write some
parse method for as described earlier by Davis.  YAML doesn't make
sense here, but JSON really does.

 Now, for something back on topic.  I had setup the readsync
 functionality, but forgot to set it to volatile.  When I did
 this, the read() method was called for syncing, but the data was
 never updated.  Is this expected behavior?  I would think that
 readsync would imply volatile.  Are there cases where readsync
 would be set, but volatile shouldn't be assumed?
 It is expected since you never said the value would change.

 Having a readsync imply volatile is sensible, but a bit magical.

 It would also eliminate the functionality of a readsync on
 non-volatile values (i.e. tell me when it is accessed even if I
 won't change it after start).
 I.e. it would make the two issues tied together rather than
 independent of each other.

OK.  Understandable.

 Lastly, there's a lot of XML happening for all these different
 devices, workers, platforms, etc. Is there a comprehensive XML
 schema that can be referenced for each of the XML data types?
 Since we use a very compact runtime parser for XML, it was not
 ever attractive to have that parser be schema-driven, since that
 would bloat it, by 2 orders of magnitude last time we checked.

 Even if there was a schema, it is far from sufficient for knowing
 whether XML is correct, so there is always more validation that
 must be coded separate from the schema.

 And schema-based parsers cannot provide semantic errors.

 That being said there are three pretty clear deficiencies in this
 area:
 1. Some people want it to all be yaml, and some want it all to be
 json :-)

 2. There is no simple standalone validation tool in OpenCPI - you
 have to "build" and get errors.

 3. The syntax-level (schema-level) error reporting is not great.

OK.  How about a different way of asking the question.

The XML files are defined across a whole slew of documentation in many
different PDF's and not coherently.

Is there a centralized document which describes each XML entity with
their properties and what they all mean? Sometimes I get errors
stating what was expected when I don't type something that is
expected.  That's all I am asking for.  I don't care about
validation.  I just want to know what's possible without having to
look mostly at examples or 5 different PDFs.

Sorry to make it overly complicated.  There is
doc/xls/OpenCPI_XML_Attributes.xlsx

It is not up to date, but if you like it, perhaps it can be.:-)

Thanks,
Brian

JSON is a great suggestion. On Mon, Apr 27, 2020 at 4:11 PM James Kulp <jek@parera.com> wrote: > On 4/27/20 4:05 PM, Brian Padalino wrote: > > On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com > > <mailto:jek@parera.com>> wrote: > > > > On 4/27/20 10:03 AM, Brian Padalino wrote: > >> On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com > >> <mailto:jek@parera.com>> wrote: > >> > >> I'll put forth a more restricted answer. The ACI can read > >> volatile > >> property values of all types, whole and in part. > >> In all cases you can get the value returned as a formatted > >> string > >> according to the documented textual format. > >> When the values being read (whole or part) are scalars or > >> arrays/sequences of scalars, they can also be read directly > >> as binary > >> values. > >> > >> These restrictions come down to the fact that OpenCPI does > >> not perform > >> data type code generation for the ACI. > >> > >> > >> Sure - that's very understandable. > >> > >> > >> One idea that has been floated is to use Google's flatbuffers > >> for this > >> purpose. > >> > >> > >> I won't comment on the binary reading part, but I would > >> definitely recommend changing the string data exchange format to > >> be JSON instead of what you have. There is a great JSON library > >> for C++11 which makes super easy to use objects and is MIT licensed: > >> > >> https://github.com/nlohmann/json > >> > > And then all the yaml devotees will start complaining! IWBNI there > > was a single textual format for everything we have. Currently our > > repo has 4....(XML, YAML, JSON, property-value).)-: > > > > > > I think my suggestion here was lost. I am not suggesting JSON be > > anywhere else, but a getPropertyJSON() method would be nice. Right > > now it spits out some formatted string which I have to write some > > parse method for as described earlier by Davis. YAML doesn't make > > sense here, but JSON really does. > > > >> Now, for something back on topic. I had setup the readsync > >> functionality, but forgot to set it to volatile. When I did > >> this, the read() method was called for syncing, but the data was > >> never updated. Is this expected behavior? I would think that > >> readsync would imply volatile. Are there cases where readsync > >> would be set, but volatile shouldn't be assumed? > > > > It is expected since you never said the value would change. > > > > Having a readsync imply volatile is sensible, but a bit magical. > > > > It would also eliminate the functionality of a readsync on > > non-volatile values (i.e. tell me when it is accessed even if I > > won't change it after start). > > I.e. it would make the two issues tied together rather than > > independent of each other. > > > > OK. Understandable. > > > >> > >> Lastly, there's a lot of XML happening for all these different > >> devices, workers, platforms, etc. Is there a comprehensive XML > >> schema that can be referenced for each of the XML data types? > > Since we use a very compact runtime parser for XML, it was not > > ever attractive to have that parser be schema-driven, since that > > would bloat it, by 2 orders of magnitude last time we checked. > > > > Even if there was a schema, it is far from sufficient for knowing > > whether XML is correct, so there is always more validation that > > must be coded separate from the schema. > > > > And schema-based parsers cannot provide semantic errors. > > > > That being said there are three pretty clear deficiencies in this > > area: > > 1. Some people want it to all be yaml, and some want it all to be > > json :-) > > > > 2. There is no simple standalone validation tool in OpenCPI - you > > have to "build" and get errors. > > > > 3. The syntax-level (schema-level) error reporting is not great. > > > > OK. How about a different way of asking the question. > > > > The XML files are defined across a whole slew of documentation in many > > different PDF's and not coherently. > > > > Is there a centralized document which describes each XML entity with > > their properties and what they all mean? Sometimes I get errors > > stating what was expected when I don't type something that is > > expected. That's all I am asking for. I don't care about > > validation. I just want to know what's possible without having to > > look mostly at examples or 5 different PDFs. > > Sorry to make it overly complicated. There is > doc/xls/OpenCPI_XML_Attributes.xlsx > > It is not up to date, but if you like it, perhaps it can be.:-) > > > > > > Thanks, > > Brian > > > > _______________________________________________ > discuss mailing list > discuss@lists.opencpi.org > http://lists.opencpi.org/mailman/listinfo/discuss_lists.opencpi.org >
JK
James Kulp
Wed, Apr 29, 2020 5:57 PM
An HTML attachment was scrubbed... URL: <http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20200429/14041e84/attachment.html>