storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ding,Dongchao" <dingdongc...@baidu.com>
Subject 答复: write to the same file in bolt?
Date Wed, 08 Jan 2014 01:00:18 GMT
Hi   ,some suggestions
You  didn’t need  to “instruct data within the same hourly tenth to the same bolt” 
 , just write   the entries within the same hourly tenth(6 min) to the same hdfs  directory
.
Because hive partition locates to one hdfs  directory ,not one hdfs file   .
thks
ding
发件人: Chen Wang [mailto:chen.apache.solr@gmail.com]
发送时间: 2014年1月8日 7:47
收件人: user@storm.incubator.apache.org
主题: write to the same file in bolt?

Hey Guys,
I am using storm to read data from our socket server, entry by entry. Each entry has a time
stamp. In my bolt, i will need to write the entries within the same hourly tenth(6 min) to
the same hdfs file, so that later i can load them to hive. (with hourly tenth 6min as the
partition).

In order to achieve that, i will either need
    1 instruct data within the same hourly tenth to the same bolt
or  2. share the same file writer for all bolts that deal with data within the same hourly
tenth.

How can I achieve this? or  if there is some other approach for this problem?
Thank you very much!
Chen


Mime
View raw message