hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Seigal <selek...@yahoo.com>
Subject incremental loads into hadoop
Date Fri, 30 Sep 2011 23:02:01 GMT

I am relatively new to Hadoop and was wondering how to do incremental
loads into HDFS.

I have a continuous stream of data flowing into a service which is
writing to an OLTP store. Due to the high volume of data, we cannot do
aggregations on the OLTP store, since this starts affecting the write

We would like to offload this processing into a Hadoop cluster, mainly
for doing aggregations/analytics.

The question is how can this continuous stream of data be
incrementally loaded and processed into Hadoop ?

Thank you,


View raw message