geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Smith (JIRA)" <>
Subject [jira] [Created] (GEODE-10) HDFS Integration
Date Thu, 07 May 2015 19:56:59 GMT
Dan Smith created GEODE-10:

             Summary: HDFS Integration
                 Key: GEODE-10
             Project: Geode
          Issue Type: Sub-task
            Reporter: Dan Smith

This is a feature that has been under development for GemFire but was not part of the initial
drop of code for geode.

HDFS Integration: Geode as a transactional layer that microbatches data out to Hadoop. This
capability makes Geode a NoSQL store that can sit on top of Hadoop and parallelize the process
of moving data from the in memory tier into Hadoop, making it very useful for capturing and
processing fast data while making it available for Hadoop jobs relatively quickly. The key
requirements being met here are

Ingest data into HDFS parallely
Cache bloom filters and allow fast lookups of individual elements
Have programmable policies for deciding what stays in memory
Roll files in HDFS
Index data that is in memory
Have expiration policies that allows the transactional set to decay out older data
Solution needs to support replicated and partitioned regions

This message was sent by Atlassian JIRA

View raw message