metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tomatophantastico <...@git.apache.org>
Subject [GitHub] metamodel issue #178: A new module for Apache Kafka "querying" using MetaMod...
Date Mon, 29 Jan 2018 22:23:34 GMT
Github user tomatophantastico commented on the issue:

    https://github.com/apache/metamodel/pull/178
  
    wrt 1) no, not required. if you got a use case, then this is fine with me.  I just think,
for getting the data from the value, you still need  a lot of domain knowledge. I just wondered
if there is a declarative way to make the value accessible. 
    
    wrt 2) i disagree that this holds for cassandra, as cassandra features a rich query api
(and cql) and beyond that a some sort of schema. in Kafka the message carries in most cases
a binary blob and you will need to iterate over all of them. 
    It is true that for all postprocessingdatacontexts (virtually all but JDBC) join and other
 operations are resource intensive, as they are performed in memory.
    
    The method you propose is fine for the infrequent case, small data set and  simple lookups.
Metamodel otoh features a rich query api, so performance issues can surface quickly.
    As this is the case with other systems, i suppose this is ok. Kafka just struck as it
is a mix of remote data locality, potentially huge size and lack of query capabilities.
    



---

Mime
View raw message