samza-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Riccomini (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SAMZA-545) Make in-memory key-value store skip serde
Date Mon, 02 Feb 2015 22:48:37 GMT

     [ https://issues.apache.org/jira/browse/SAMZA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Riccomini updated SAMZA-545:
----------------------------------
    Description: 
SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the layering in
Samza's KV-store APIs, the in-memory store still holds raw bytes, and the Serde is used to
transform objects back into POJOs. On the read-side, it is unnecessary to deserialize the
bytes back into an object. The in-memory KV store should just hold the raw object.

Semantically, this does change the behavior of the KV-store a bit, when using in-memory stores.
If an object is mutated after it's been written to an in-memory store, and then store.get
is called, the mutated object will be returned. This is not the case with regular (LevelDB/RocksDB)
KV-stores.

Writes will still require serializing the object if a changelog is attached. If a changelog
is not attached, then I'd argue that the in-memory KV store should not be used at all, and
a simple HashMap should be used instead.

When updating the code, we should be mindful to keep the API as clean as possible, while shifting
the layers around.

  was:
SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the layering in
Samza's KV-store APIs, the in-memory store still holds raw bytes, and the Serde is used to
transform objects back into POJOs. On the read-side, it is unnecessary to deserialize the
byte back into an object. The in-memory KV store should just hold the raw object.

Semantically, this does change the behavior of the KV-store a bit, when using in-memory stores.
If an object is mutated after it's been written to an in-memory store, and then store.get
is called, the mutated object will be returned. This is not the case with regular (LevelDB/RocksDB)
KV-stores.

Writes will still require serializing the object if a changelog is attached. If a changelog
is not attached, then I'd argue that the in-memory KV store should not be used at all, and
a simple HashMap should be used instead.

When updating the code, we should be mindful to keep the API as clean as possible, while shifting
the layers around.


> Make in-memory key-value store skip serde
> -----------------------------------------
>
>                 Key: SAMZA-545
>                 URL: https://issues.apache.org/jira/browse/SAMZA-545
>             Project: Samza
>          Issue Type: Bug
>          Components: kv
>    Affects Versions: 0.9.0
>            Reporter: Chris Riccomini
>
> SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the layering
in Samza's KV-store APIs, the in-memory store still holds raw bytes, and the Serde is used
to transform objects back into POJOs. On the read-side, it is unnecessary to deserialize the
bytes back into an object. The in-memory KV store should just hold the raw object.
> Semantically, this does change the behavior of the KV-store a bit, when using in-memory
stores. If an object is mutated after it's been written to an in-memory store, and then store.get
is called, the mutated object will be returned. This is not the case with regular (LevelDB/RocksDB)
KV-stores.
> Writes will still require serializing the object if a changelog is attached. If a changelog
is not attached, then I'd argue that the in-memory KV store should not be used at all, and
a simple HashMap should be used instead.
> When updating the code, we should be mindful to keep the API as clean as possible, while
shifting the layers around.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message