metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kaspersorensen <...@git.apache.org>
Subject [GitHub] metamodel issue #178: A new module for Apache Kafka "querying" using MetaMod...
Date Mon, 29 Jan 2018 16:14:32 GMT
Github user kaspersorensen commented on the issue:

    https://github.com/apache/metamodel/pull/178
  
    To address your two discussion points:
    
    1. Yes it is very typical to store some data structure in Kafka. I definitely think that
a natural improvement here would be to extend the module here with more pluggable serialization
support. I wonder if the already existing Converters utils would work. I think I’ll try
that first.
    Btw this is not very different from how our Cassandra module works today since Cassandra
also exposes just the stored bytes as the base of its API.
    
    2. Certainly some queries would not work well here. Using ORDER BY on a stream means that
we have to exhaust the stream first. But OTOH many queries will work quite well and that’s
what I was looking for. It does do streaming of the records out to the client, so it won’t
have to materialize everything in memory to serve those queries.


---

Mime
View raw message