samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagadish Venkatraman <jagadish1...@gmail.com>
Subject Re: Using json serde with KeyValueStore
Date Tue, 08 Nov 2016 08:02:50 GMT
Good to know!

Thanks,
Jagadish

On Tue, Nov 8, 2016 at 12:01 AM, Raj raj <rajlistuser@gmail.com> wrote:

> Thanks!
>
> It is working as expected.
>
> raj
> Thanks and Regards,
>
> Raj
>
>
> On Sat, Nov 5, 2016 at 11:24 PM, Jagadish Venkatraman
> <jagadish1989@gmail.com> wrote:
> > You can write a small test job and verify this! I believe the serde
> should
> > probably be able to serialize it correctly.
> >
> > You can define the KeyValueStore to be typed as <String,Object>.
> >
> > - You can put the LHM directly into the store.
> > -  When you do a get on the store, you should probably be able re-cast it
> > back to an LHM.
> >
> > String/json serialization is not necessarily performant for all
> > cases. There maybe other efficient serialization formats like protobuffs/
> > avro depending on the use case. Should you want to write your own
> > serialization have a look at the SerdeFactory interface.
> >
> > Thanks
> > Jag
> >
> > On Saturday, November 5, 2016, Raj raj <rajlistuser@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> I am starting out with Samza and my test program is able to read a
> >> message from Kafka. My next task is to write the message to KeyValue
> >> store. The message is formatted as Json in Kafka and I am using json
> >> serde as my msg serde. The incoming message is available to me as a
> >> LinkedHashMap when I call envelope.getMessage(). i.e. serde
> >> automatically converted the json string to a LinkedHashMap.
> >>
> >> Now I want to store this to KeyValueStore. Will using a json serde for
> >> stores.test-container.msg.serde automatically convert the
> >> LinkedHashMap to a format suitable for storing in the KeyValueStore?
> >> i.e. the reverse process of what it did when I called
> >> envelope.getMessage().
> >>
> >> If it is not possible what is the best practice of storing a
> >> LinkedHashMap in KeyStoreValue?
> >>
> >> I am pasting the current code I am working with in pastebin for
> >> reference, there isn't much in the code, just the boilerplate stuff.
> >>
> >> test-container.properties : http://pastebin.com/dkbnjtNh
> >> TestContainer.java : http://pastebin.com/93ZBgeWa
> >>
> >> Thanks and Regards,
> >>
> >> Raj
> >>
> >
> >
> > --
> > Sent from my iphone.
>



-- 
Jagadish V,
Graduate Student,
Department of Computer Science,
Stanford University

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message