Accessing Complex Properties (struct) of A Worker

BP
Brian Padalino
Wed, Apr 29, 2020 6:12 PM

On Wed, Apr 29, 2020 at 1:57 PM James Kulp jek@parera.com wrote:

The simplest thing I can think of (i.e. short term easy/doable), is an
option when property values are being retrieved as text, to have an option
to produce a json string rather than the "native" text format.  This
requires no library.

Just curious, why are you against the MIT licensed library I posted:

https://github.com/nlohmann/json

It's a single header file you include, so no extra recompilation and it's
very liberally licensed so you can modify it all you want without any
issues.

Also not sure why getPropertyJSON() would be a bad addition to the API?  I
guess multiple ways to get text could be problematic or ambiguous?

Since the native text format is more strongly typed, its a bit harder to
go in the other direction, and would require choosing a library and dealing
with various type ambiguities.
Just a bit more work.

It also seems very well documented, like the basic usage here for going
back and forth between types:

https://github.com/nlohmann/json#basic-usage

Either way, I appreciate the answers you guys have given and the fact I can
retrieve the whole string in a single read is great.

Thank you very much.

Brian

On Wed, Apr 29, 2020 at 1:57 PM James Kulp <jek@parera.com> wrote: > The simplest thing I can think of (i.e. short term easy/doable), is an > option when property values are being retrieved as text, to have an option > to produce a json string rather than the "native" text format. This > requires no library. > Just curious, why are you against the MIT licensed library I posted: https://github.com/nlohmann/json It's a single header file you include, so no extra recompilation and it's very liberally licensed so you can modify it all you want without any issues. Also not sure why getPropertyJSON() would be a bad addition to the API? I guess multiple ways to get text could be problematic or ambiguous? > > Since the native text format is more strongly typed, its a bit harder to > go in the other direction, and would require choosing a library and dealing > with various type ambiguities. > Just a bit more work. > It also seems very well documented, like the basic usage here for going back and forth between types: https://github.com/nlohmann/json#basic-usage Either way, I appreciate the answers you guys have given and the fact I can retrieve the whole string in a single read is great. Thank you very much. Brian
JK
James Kulp
Wed, Apr 29, 2020 6:56 PM

The simplest thing I can think of (i.e. short term easy/doable), is an
option when property values are being retrieved as text, to have an
option to produce a json string rather than the "native" text format. 
This requires no library.

Since the native text format is more strongly typed, its a bit harder to
go in the other direction, and would require choosing a library and
dealing with various type ambiguities.
Just a bit more work.

On 4/27/20 4:48 PM, Chris Hinkey wrote:

JSON is a great suggestion.

On Mon, Apr 27, 2020 at 4:11 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:

 On 4/27/20 4:05 PM, Brian Padalino wrote:

On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com

 <mailto:jek@parera.com>

<mailto:jek@parera.com mailto:jek@parera.com>> wrote:

     On 4/27/20 10:03 AM, Brian Padalino wrote:

     On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com

 <mailto:jek@parera.com>

     <mailto:jek@parera.com mailto:jek@parera.com>> wrote:

         I'll put forth a more restricted answer.  The ACI can read
         volatile
         property values of all types, whole and in part.
         In all cases you can get the value returned as a formatted
         string
         according to the documented textual format.
         When the values being read (whole or part) are scalars or
         arrays/sequences of scalars, they can also be read directly
         as binary
         values.

         These restrictions come down to the fact that OpenCPI does
         not perform
         data type code generation for the ACI.

     Sure - that's very understandable.

         One idea that has been floated is to use Google's

 flatbuffers

         for this
         purpose.

     I won't comment on the binary reading part, but I would
     definitely recommend changing the string data exchange

 format to

     be JSON instead of what you have.  There is a great JSON

 library

     for C++11 which makes super easy to use objects and is MIT

 licensed:

     And then all the yaml devotees will start complaining! IWBNI

 there

     was a single textual format for everything we have.

 Currently our

     repo has 4....(XML, YAML, JSON, property-value).)-:

I think my suggestion here was lost.  I am not suggesting JSON be
anywhere else, but a getPropertyJSON() method would be nice.  Right
now it spits out some formatted string which I have to write some
parse method for as described earlier by Davis.  YAML doesn't make
sense here, but JSON really does.

     Now, for something back on topic.  I had setup the readsync
     functionality, but forgot to set it to volatile. When I did
     this, the read() method was called for syncing, but the

 data was

     never updated.  Is this expected behavior?  I would think that
     readsync would imply volatile.  Are there cases where readsync
     would be set, but volatile shouldn't be assumed?

     It is expected since you never said the value would change.

     Having a readsync imply volatile is sensible, but a bit magical.

     It would also eliminate the functionality of a readsync on
     non-volatile values (i.e. tell me when it is accessed even if I
     won't change it after start).
     I.e. it would make the two issues tied together rather than
     independent of each other.

 OK.  Understandable.

     Lastly, there's a lot of XML happening for all these different
     devices, workers, platforms, etc. Is there a comprehensive XML
     schema that can be referenced for each of the XML data types?

     Since we use a very compact runtime parser for XML, it was not
     ever attractive to have that parser be schema-driven, since that
     would bloat it, by 2 orders of magnitude last time we checked.

     Even if there was a schema, it is far from sufficient for

 knowing

     whether XML is correct, so there is always more validation that
     must be coded separate from the schema.

     And schema-based parsers cannot provide semantic errors.

     That being said there are three pretty clear deficiencies in

 this

     area:
     1. Some people want it to all be yaml, and some want it all

 to be

     json :-)

     2. There is no simple standalone validation tool in OpenCPI

 - you

     have to "build" and get errors.

     3. The syntax-level (schema-level) error reporting is not great.

OK.  How about a different way of asking the question.

The XML files are defined across a whole slew of documentation

 in many

different PDF's and not coherently.

Is there a centralized document which describes each XML entity

 with

their properties and what they all mean? Sometimes I get errors
stating what was expected when I don't type something that is
expected.  That's all I am asking for.  I don't care about
validation.  I just want to know what's possible without having to
look mostly at examples or 5 different PDFs.

 Sorry to make it overly complicated.  There is
 doc/xls/OpenCPI_XML_Attributes.xlsx

 It is not up to date, but if you like it, perhaps it can be.:-)

Thanks,
Brian

 _______________________________________________
 discuss mailing list
 discuss@lists.opencpi.org <mailto:discuss@lists.opencpi.org>
 http://lists.opencpi.org/mailman/listinfo/discuss_lists.opencpi.org
The simplest thing I can think of (i.e. short term easy/doable), is an option when property values are being retrieved as text, to have an option to produce a json string rather than the "native" text format.  This requires no library. Since the native text format is more strongly typed, its a bit harder to go in the other direction, and would require choosing a library and dealing with various type ambiguities. Just a bit more work. On 4/27/20 4:48 PM, Chris Hinkey wrote: > JSON is a great suggestion. > > On Mon, Apr 27, 2020 at 4:11 PM James Kulp <jek@parera.com > <mailto:jek@parera.com>> wrote: > > On 4/27/20 4:05 PM, Brian Padalino wrote: > > On Mon, Apr 27, 2020 at 3:48 PM James Kulp <jek@parera.com > <mailto:jek@parera.com> > > <mailto:jek@parera.com <mailto:jek@parera.com>>> wrote: > > > >     On 4/27/20 10:03 AM, Brian Padalino wrote: > >>     On Sat, Apr 25, 2020 at 8:13 AM James Kulp <jek@parera.com > <mailto:jek@parera.com> > >>     <mailto:jek@parera.com <mailto:jek@parera.com>>> wrote: > >> > >>         I'll put forth a more restricted answer.  The ACI can read > >>         volatile > >>         property values of all types, whole and in part. > >>         In all cases you can get the value returned as a formatted > >>         string > >>         according to the documented textual format. > >>         When the values being read (whole or part) are scalars or > >>         arrays/sequences of scalars, they can also be read directly > >>         as binary > >>         values. > >> > >>         These restrictions come down to the fact that OpenCPI does > >>         not perform > >>         data type code generation for the ACI. > >> > >> > >>     Sure - that's very understandable. > >> > >> > >>         One idea that has been floated is to use Google's > flatbuffers > >>         for this > >>         purpose. > >> > >> > >>     I won't comment on the binary reading part, but I would > >>     definitely recommend changing the string data exchange > format to > >>     be JSON instead of what you have.  There is a great JSON > library > >>     for C++11 which makes super easy to use objects and is MIT > licensed: > >> > >> https://github.com/nlohmann/json > >> > >     And then all the yaml devotees will start complaining! IWBNI > there > >     was a single textual format for everything we have. > Currently our > >     repo has 4....(XML, YAML, JSON, property-value).)-: > > > > > > I think my suggestion here was lost.  I am not suggesting JSON be > > anywhere else, but a getPropertyJSON() method would be nice.  Right > > now it spits out some formatted string which I have to write some > > parse method for as described earlier by Davis.  YAML doesn't make > > sense here, but JSON really does. > > > >>     Now, for something back on topic.  I had setup the readsync > >>     functionality, but forgot to set it to volatile. When I did > >>     this, the read() method was called for syncing, but the > data was > >>     never updated.  Is this expected behavior?  I would think that > >>     readsync would imply volatile.  Are there cases where readsync > >>     would be set, but volatile shouldn't be assumed? > > > >     It is expected since you never said the value would change. > > > >     Having a readsync imply volatile is sensible, but a bit magical. > > > >     It would also eliminate the functionality of a readsync on > >     non-volatile values (i.e. tell me when it is accessed even if I > >     won't change it after start). > >     I.e. it would make the two issues tied together rather than > >     independent of each other. > > > >  OK.  Understandable. > > > >> > >>     Lastly, there's a lot of XML happening for all these different > >>     devices, workers, platforms, etc. Is there a comprehensive XML > >>     schema that can be referenced for each of the XML data types? > >     Since we use a very compact runtime parser for XML, it was not > >     ever attractive to have that parser be schema-driven, since that > >     would bloat it, by 2 orders of magnitude last time we checked. > > > >     Even if there was a schema, it is far from sufficient for > knowing > >     whether XML is correct, so there is always more validation that > >     must be coded separate from the schema. > > > >     And schema-based parsers cannot provide semantic errors. > > > >     That being said there are three pretty clear deficiencies in > this > >     area: > >     1. Some people want it to all be yaml, and some want it all > to be > >     json :-) > > > >     2. There is no simple standalone validation tool in OpenCPI > - you > >     have to "build" and get errors. > > > >     3. The syntax-level (schema-level) error reporting is not great. > > > > OK.  How about a different way of asking the question. > > > > The XML files are defined across a whole slew of documentation > in many > > different PDF's and not coherently. > > > > Is there a centralized document which describes each XML entity > with > > their properties and what they all mean? Sometimes I get errors > > stating what was expected when I don't type something that is > > expected.  That's all I am asking for.  I don't care about > > validation.  I just want to know what's possible without having to > > look mostly at examples or 5 different PDFs. > > Sorry to make it overly complicated.  There is > doc/xls/OpenCPI_XML_Attributes.xlsx > > It is not up to date, but if you like it, perhaps it can be.:-) > > > > > > Thanks, > > Brian > > > > _______________________________________________ > discuss mailing list > discuss@lists.opencpi.org <mailto:discuss@lists.opencpi.org> > http://lists.opencpi.org/mailman/listinfo/discuss_lists.opencpi.org >
JK
James Kulp
Wed, Apr 29, 2020 7:12 PM
An HTML attachment was scrubbed... URL: <http://lists.opencpi.org/pipermail/discuss_lists.opencpi.org/attachments/20200429/77520514/attachment.html>
JK
James Kulp
Wed, Apr 29, 2020 7:20 PM

On 4/29/20 2:12 PM, Brian Padalino wrote:

On Wed, Apr 29, 2020 at 1:57 PM James Kulp <jek@parera.com
mailto:jek@parera.com> wrote:

 The simplest thing I can think of (i.e. short term easy/doable),
 is an option when property values are being retrieved as text, to
 have an option to produce a json string rather than the "native"
 text format.  This requires no library.

Just curious, why are you against the MIT licensed library I posted:

https://github.com/nlohmann/json

It's a single header file you include, so no extra recompilation and
it's very liberally licensed so you can modify it all you want
without any issues.

Also not sure why getPropertyJSON() would be a bad addition to the
API?  I guess multiple ways to get text could be problematic or ambiguous?

That is pretty much what was proposed.  There is no being "against" that
library, its just that the binary-to-text conversion does not need a
library since given the simplicity of JSON syntax, it wouldn't help
since its just a simple textual serialization of existing internal data
structure.  Plus it would allow developers to use whatever library they
wanted.

OTOH, going the other way would be best done with a library, and the one
you like seems just fine, and binary-to-binary conversion could easily
use the library, which would then get the serialization for free.

 Since the native text format is more strongly typed, its a bit
 harder to go in the other direction, and would require choosing a
 library and dealing with various type ambiguities.
 Just a bit more work.

It also seems very well documented, like the basic usage here for
going back and forth between types:

https://github.com/nlohmann/json#basic-usage

Either way, I appreciate the answers you guys have given and the fact
I can retrieve the whole string in a single read is great.

Thank you very much.

Brian

On 4/29/20 2:12 PM, Brian Padalino wrote: > On Wed, Apr 29, 2020 at 1:57 PM James Kulp <jek@parera.com > <mailto:jek@parera.com>> wrote: > > The simplest thing I can think of (i.e. short term easy/doable), > is an option when property values are being retrieved as text, to > have an option to produce a json string rather than the "native" > text format.  This requires no library. > > > Just curious, why are you against the MIT licensed library I posted: > > https://github.com/nlohmann/json > > It's a single header file you include, so no extra recompilation and > it's very liberally licensed so you can modify it all you want > without any issues. > > Also not sure why getPropertyJSON() would be a bad addition to the > API?  I guess multiple ways to get text could be problematic or ambiguous? That is pretty much what was proposed.  There is no being "against" that library, its just that the binary-to-text conversion does not need a library since given the simplicity of JSON syntax, it wouldn't help since its just a simple textual serialization of existing internal data structure.  Plus it would allow developers to use whatever library they wanted. OTOH, going the other way would be best done with a library, and the one you like seems just fine, and binary-to-binary conversion could easily use the library, which would then get the serialization for free. > > Since the native text format is more strongly typed, its a bit > harder to go in the other direction, and would require choosing a > library and dealing with various type ambiguities. > Just a bit more work. > > > It also seems very well documented, like the basic usage here for > going back and forth between types: > > https://github.com/nlohmann/json#basic-usage > > Either way, I appreciate the answers you guys have given and the fact > I can retrieve the whole string in a single read is great. > > Thank you very much. > > Brian