plc4x-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Feinauer <j.feina...@pragmaticminds.de>
Subject Re: Antwort: Re: Antwort: Re: Antwort: Re: Antwort: Re: Anyone got experience with "protocol buffers" or DFDL (for generation of the message (de)serialization code)?
Date Wed, 09 Jan 2019 17:16:01 GMT
Hi Chris,

I'm not sure if I got it right, but what we would need is a way to generate classes in multiple
languages based on a message definition, or?
So we say something like
Bit, byte, byte, bit
And it generates a class (or struct?) which deserializes / serializes to that, or?

Sorry for that (probably) dump question but I got a bit confused by this protobuf / thrift
approach.

Julian

Am 09.01.19, 17:49 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:

    Hi Sebastian,
    
    Well in general the difference between a read and a write in S7 is one byte value ;-)
    The overall structure is somewhat identical.
    
    Chris
    
    
    
    Von: Sebastian Wiendl <SWiendl@bhs-corrugated.de>
    Antworten an: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum: Mittwoch, 9. Januar 2019 um 17:21
    An: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Betreff: Antwort: Re: Antwort: Re: Antwort: Re: Antwort: Re: Anyone got experience with
"protocol buffers" or DFDL (for generation of the message (de)serialization code)?
    
    The developer's struggle: Which foot to shoot yourself in. ;-)
    
    I have no detailed knowledge about the protocols you want to serialize/deserialize - maybe
they are simple enough that the mentioned early serialization features of Kaitai are sufficient.
If not, I wouldn't recommend using it.
    
    Another aspect might be the anticipated distribution of read and write workloads of a
typical PLC4X application - if its mostly reads it might still be worth a try, because you
save a lot of (implementation) effort on the deserialization which frees up resources for
serialization or other things in general.
    
    Mit freundlichen Grüßen
    Kind regards
    
    Sebastian Wiendl
    DSE / Digital Solutions Software Engineer
    Phone: +49 9605 919 - 9341
    E-Mail: SWiendl@bhs-corrugated.de
    Internet: http://www.bhs-world.com
    ________________________________
     [cid:_1_13C7BE3013C7BA340059D4ACC125837D]
    
    BHS Corrugated Maschinen- und Anlagenbau GmbH
    Paul-Engel-Straße 1
    92729 WEIHERHAMMER
    GERMANY
    
    Management: Christian Engel, Lars Engel, Norbert Städele
    Registered at Amtsgericht Weiden, HR B 1320
    
    [cid:_2_13C7CB4413C7C7340059D4ACC125837D]
    
    
    ________________________________
    
    
    Diese Nachricht ist nur für den Empfänger bestimmt, da sie persönliche und/oder rechtlich
geschützte Informationen und vertrauliche Informationen enthält.
    Der Inhalt darf an keinen anderen außer an den Empfänger weitergeleitet werden. Unbefugte
Empfänger werden gebeten, diese Vertraulichkeit zu wahren und den Absender sofort von diesem
Übertragungsfehler in Kenntnis zu setzen.
    Vernichten Sie diese Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
dieser Nachricht ist nicht gestattet.
    
    This message is intended for the addressee only as it contains private and/or privileged
and confidential information.
    The contents are not to be disclosed to anyone else than the addressee. Unauthorized recipients
are requested to comply with the above and to inform the sender immediately of any errors
in transmission and destroy this e-mail.
    Any unauthorized copying, disclosure or distribution of the material in this e-mail is
strictly prohibited.
    
    
    
    
    Von:        "Christofer Dutz" <christofer.dutz@c-ware.de>
    An:        "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum:        09.01.2019 17:09
    Betreff:        Re: Antwort: Re: Antwort: Re: Antwort: Re: Anyone got experience with
"protocol buffers" or DFDL (for generation of the message (de)serialization code)?
    ________________________________
    
    
    
    Hi Sebastian,
    
    yeah … I asked in the Incubator list and they told me it would be safe to use, if the
output was not GPLed …
    
    so I contacted the maintainer of the project and simply asked him.
    He told me that the output would match the input license. So if our definitions would
be Apache 2.0, so would the output … so it seems we would be safe and able to use that.
    However he also told me this:
    
    “That's great news! Please note, however, that serialization is in its
    early stages in Kaitai Struct, so it might be no exactly up to your
    expectations.“
    
    So I don’t know if we should go down that path … right now … :-/
    
    Chris
    
    
    
    Von: Sebastian Wiendl <SWiendl@bhs-corrugated.de>
    Antworten an: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum: Mittwoch, 9. Januar 2019 um 16:46
    An: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Betreff: Antwort: Re: Antwort: Re: Antwort: Re: Anyone got experience with "protocol buffers"
or DFDL (for generation of the message (de)serialization code)?
    
    Another thing to consider: Last time I checked Kaitai only supported deserialization (https://github.com/kaitai-io/kaitai_struct/issues/27).
Might be inconvenient to introduce another framework for serialization...
    
    Mit freundlichen Grüßen
    Kind regards
    
    Sebastian Wiendl
    DSE / Digital Solutions Software Engineer
    Phone: +49 9605 919 - 9341
    E-Mail: SWiendl@bhs-corrugated.de
    Internet: http://www.bhs-world.com<http://www.bhs-world.com/>
    ________________________________
    [cid:_1_147DDC68147DD8840056A390C125837D]
    
    BHS Corrugated Maschinen- und Anlagenbau GmbH
    Paul-Engel-Straße 1
    92729 WEIHERHAMMER
    GERMANY
    
    Management: Christian Engel, Lars Engel, Norbert Städele
    Registered at Amtsgericht Weiden, HR B 1320
    
    [cid:_2_147DE97C147DE56C0056A390C125837D]
    
    
    ________________________________
    
    
    Diese Nachricht ist nur für den Empfänger bestimmt, da sie persönliche und/oder rechtlich
geschützte Informationen und vertrauliche Informationen enthält.
    Der Inhalt darf an keinen anderen außer an den Empfänger weitergeleitet werden. Unbefugte
Empfänger werden gebeten, diese Vertraulichkeit zu wahren und den Absender sofort von diesem
Übertragungsfehler in Kenntnis zu setzen.
    Vernichten Sie diese Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
dieser Nachricht ist nicht gestattet.
    
    This message is intended for the addressee only as it contains private and/or privileged
and confidential information.
    The contents are not to be disclosed to anyone else than the addressee. Unauthorized recipients
are requested to comply with the above and to inform the sender immediately of any errors
in transmission and destroy this e-mail.
    Any unauthorized copying, disclosure or distribution of the material in this e-mail is
strictly prohibited.
    
    
    
    
    Von:        "Christofer Dutz" <christofer.dutz@c-ware.de>
    An:        "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum:        09.01.2019 15:57
    Betreff:        Re: Antwort: Re: Antwort: Re: Anyone got experience with "protocol buffers"
or DFDL (for generation of the message (de)serialization code)?
    ________________________________
    
    
    
    The thing is that we would actually not be bundling that … the compiler is only used
during the build and not at runtime.
    I am currently double-checking this with the others in the incubator mailing-list.
    
    So it’s not completely impossible … let’s see what the others have to say about
it.
    
    But I agree … it does look as if it would suit our needs.
    
    Chris
    
    
    Von: Sebastian Wiendl <SWiendl@bhs-corrugated.de>
    Antworten an: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum: Mittwoch, 9. Januar 2019 um 15:42
    An: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Betreff: Antwort: Re: Antwort: Re: Anyone got experience with "protocol buffers" or DFDL
(for generation of the message (de)serialization code)?
    
    Yeah, i just checked... bummer the compiler is GPLed...
    
    The project itself has some really good ideas how to handle binary parsing. I used it
prototypically in decoding a proprietary UDP message format and it worked great.
    
    Mit freundlichen Grüßen
    Kind regards
    
    Sebastian Wiendl
    DSE / Digital Solutions Software Engineer
    Phone: +49 9605 919 - 9341
    E-Mail: SWiendl@bhs-corrugated.de
    Internet: http://www.bhs-world.com<http://www.bhs-world.com/><http://www.bhs-world.com/>
    ________________________________
    [cid:_1_147D3C84147D38880050BA3BC125837D]
    
    BHS Corrugated Maschinen- und Anlagenbau GmbH
    Paul-Engel-Straße 1
    92729 WEIHERHAMMER
    GERMANY
    
    Management: Christian Engel, Lars Engel, Norbert Städele
    Registered at Amtsgericht Weiden, HR B 1320
    
    [cid:_2_147D4998147D45880050BA3BC125837D]
    
    
    ________________________________
    
    
    Diese Nachricht ist nur für den Empfänger bestimmt, da sie persönliche und/oder rechtlich
geschützte Informationen und vertrauliche Informationen enthält.
    Der Inhalt darf an keinen anderen außer an den Empfänger weitergeleitet werden. Unbefugte
Empfänger werden gebeten, diese Vertraulichkeit zu wahren und den Absender sofort von diesem
Übertragungsfehler in Kenntnis zu setzen.
    Vernichten Sie diese Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
dieser Nachricht ist nicht gestattet.
    
    This message is intended for the addressee only as it contains private and/or privileged
and confidential information.
    The contents are not to be disclosed to anyone else than the addressee. Unauthorized recipients
are requested to comply with the above and to inform the sender immediately of any errors
in transmission and destroy this e-mail.
    Any unauthorized copying, disclosure or distribution of the material in this e-mail is
strictly prohibited.
    
    
    
    
    Von:        "Christofer Dutz" <christofer.dutz@c-ware.de>
    An:        "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum:        09.01.2019 15:37
    Betreff:        Re: Antwort: Re: Anyone got experience with "protocol buffers" or DFDL
(for generation of the message (de)serialization code)?
    ________________________________
    
    
    
    Hi Sebasitan,
    
    thanks for that pointer … unfortunately I think we might be unable to use this option
as the compiler is GPLv3 licensed … that is a category X license.
    Even if the compiler is not bundled with our software or used at runtime (will probably
only need it at compile-time), I doubt we would be allowed to use it.
    
    But still I’ll look into it …
    
    Chris
    
    Von: Sebastian Wiendl <SWiendl@bhs-corrugated.de>
    Antworten an: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum: Mittwoch, 9. Januar 2019 um 15:18
    An: "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Betreff: Antwort: Re: Anyone got experience with "protocol buffers" or DFDL (for generation
of the message (de)serialization code)?
    
    Hi,
    
    maybe the Kaitai Struct project (https://kaitai.io/) can help you achieve this?
    
    Mit freundlichen Grüßen
    Kind regards
    
    Sebastian Wiendl
    DSE / Digital Solutions Software Engineer
    Phone: +49 9605 919 - 9341
    E-Mail: SWiendl@bhs-corrugated.de
    Internet: http://www.bhs-world.com<http://www.bhs-world.com/><http://www.bhs-world.com/><http://www.bhs-world.com/>
    ________________________________
    [cid:_1_0DB11BC80DB117CC004E9C08C125837D]
    
    BHS Corrugated Maschinen- und Anlagenbau GmbH
    Paul-Engel-Straße 1
    92729 WEIHERHAMMER
    GERMANY
    
    Management: Christian Engel, Lars Engel, Norbert Städele
    Registered at Amtsgericht Weiden, HR B 1320
    
    [cid:_2_0DB128DC0DB124CC004E9C08C125837D]
    
    
    ________________________________
    
    
    Diese Nachricht ist nur für den Empfänger bestimmt, da sie persönliche und/oder rechtlich
geschützte Informationen und vertrauliche Informationen enthält.
    Der Inhalt darf an keinen anderen außer an den Empfänger weitergeleitet werden. Unbefugte
Empfänger werden gebeten, diese Vertraulichkeit zu wahren und den Absender sofort von diesem
Übertragungsfehler in Kenntnis zu setzen.
    Vernichten Sie diese Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
dieser Nachricht ist nicht gestattet.
    
    This message is intended for the addressee only as it contains private and/or privileged
and confidential information.
    The contents are not to be disclosed to anyone else than the addressee. Unauthorized recipients
are requested to comply with the above and to inform the sender immediately of any errors
in transmission and destroy this e-mail.
    Any unauthorized copying, disclosure or distribution of the material in this e-mail is
strictly prohibited.
    
    
    
    
    Von:        "Christofer Dutz" <christofer.dutz@c-ware.de>
    An:        "dev@plc4x.apache.org" <dev@plc4x.apache.org>
    Datum:        09.01.2019 15:06
    Betreff:        Re: Anyone got experience with "protocol buffers" or DFDL (for generation
of the message (de)serialization code)?
    ________________________________
    
    
    
    Hi all,
    
    Ok ... so protobuf seems to be semi-ideal ...
    
    It seems that you can use it to model the structure of data. Protobuf is good for generating
model classes, parsers and serializers for a given model ... the binary data-format is a result
of this.
    
    We want the opposite: We want to generate a model from a known output data-format. In
general this could be somehow achieved with protobuf, however it is very difficult to produce
the definition in a way that it is able to parse a given data format.
    For example simply outputting one byte seems to be problematic. I was able to somehow
hack an enum and provide some extension to allow providing code values, but we don't have
the level of control we would need to and the result is not very readable.
    I was able to quite easily setup the maven build to generate java code for parsing and
serializing a model ... so that was good.
    
    DFDL looks as if it's ideal for describing the data format, however I couldn't find tooling
to generate model, parser and serializer from a DFDL definition. I subscribed to our brother
incubating project Daffodil and asked on their list ... perhaps I have to get my hands dirty
and implement the maven plugin and code generators as part of that project ... I am hoping
not having to do that.
    
    I'll check out Thrift in parallel  ;-)
    
    
    Chris
    
    
    
    Am 09.01.19, 11:19 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:
    
    From my first look at thrift some time ago, that's more about API and not about the actual
payload, is it?
    
    How about I try to do a protobuf version of the "s7-protocol" and you give thrift a try?
Another option would be the DFDL option.
    
    Chris
    
    Am 09.01.19, 11:13 schrieb "Julian Feinauer" <j.feinauer@pragmaticminds.de>:
    
        Hi Chris,
    
        we worked (and work) with Thrift [1] at several places.
        Thrift is a strong contender to protobuf and both have their specific advantages and
disadvantages.
        Perhaps I would prefer Thrift as it comes from the Apache Ecosystm (and supports more
langauges) but generally, Tim can say more about working with Thrift.
    
        Best
        Julian
    
        [1] https://thrift.apache.org/
    
        Am 09.01.19, 10:45 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:
    
            Hi all,
    
            while I’m currently working on refactoring the S7 driver to a simpler structure
so we can convert it to other languages more easily. A colleague of mine pointed me to protobuf/protocol
buffers from google [1]
            From a quick look at it, it does seem as if it could suit our needs quite nicely.
I would like to try out if it’s possible to model the S7 data structures in this way. If
it works we could eventually quickly create something that serializes/deserializes given data
in any language …
    
            It seems to be a lot simpler than the DFDL [2] I was thinking of, so guess we
have to find out if it has all the capabilities we need.
    
            Any thoughts?
    
            Chris
    
    
    
    
            [1] https://developers.google.com/protocol-buffers/docs/javatutorial
            [2] https://en.wikipedia.org/wiki/Data_Format_Description_Language
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

Mime
View raw message