samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagadish Venkatraman <jagadish1...@gmail.com>
Subject Re: Using json serde with KeyValueStore
Date Sat, 05 Nov 2016 17:54:55 GMT
You can write a small test job and verify this! I believe the serde should
probably be able to serialize it correctly.

You can define the KeyValueStore to be typed as <String,Object>.

- You can put the LHM directly into the store.
-  When you do a get on the store, you should probably be able re-cast it
back to an LHM.

String/json serialization is not necessarily performant for all
cases. There maybe other efficient serialization formats like protobuffs/
avro depending on the use case. Should you want to write your own
serialization have a look at the SerdeFactory interface.

Thanks
Jag

On Saturday, November 5, 2016, Raj raj <rajlistuser@gmail.com> wrote:

> Hi,
>
> I am starting out with Samza and my test program is able to read a
> message from Kafka. My next task is to write the message to KeyValue
> store. The message is formatted as Json in Kafka and I am using json
> serde as my msg serde. The incoming message is available to me as a
> LinkedHashMap when I call envelope.getMessage(). i.e. serde
> automatically converted the json string to a LinkedHashMap.
>
> Now I want to store this to KeyValueStore. Will using a json serde for
> stores.test-container.msg.serde automatically convert the
> LinkedHashMap to a format suitable for storing in the KeyValueStore?
> i.e. the reverse process of what it did when I called
> envelope.getMessage().
>
> If it is not possible what is the best practice of storing a
> LinkedHashMap in KeyStoreValue?
>
> I am pasting the current code I am working with in pastebin for
> reference, there isn't much in the code, just the boilerplate stuff.
>
> test-container.properties : http://pastebin.com/dkbnjtNh
> TestContainer.java : http://pastebin.com/93ZBgeWa
>
> Thanks and Regards,
>
> Raj
>


-- 
Sent from my iphone.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message