hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: [jira] [Updated] (HAMA-883) [Research Task] Massive log event aggregation in real time using Apache Hama
Date Wed, 05 Mar 2014 04:25:20 GMT
Why not?

Sent from my iPhone

> On 2014. 3. 5., at 오후 1:09, Yexi Jiang <yexijiang@gmail.com> wrote:
> 
> Yes, currently Hama does not support streaming input and streaming output.
> That's why currently it is not a natural choice for people with real time
> computing needs.
> 
> Do we really need to make Hama to support the real time computing? In that
> case, we need to compete with Storm...
> 
> 
> 2014-03-04 22:58 GMT-05:00 Chia-Hung Lin <clin4j@googlemail.com>:
> 
>> I used Twitter Storm previously. Storm is an excellent framework in
>> real time processing.
>> 
>> Considering Hama in real time tasks, the framework in my opinion need
>> to decouple io from hdfs so that the source/ input is not restricted
>> to just hdfs.
>> 
>>> On 5 March 2014 09:30, Yexi Jiang <yexijiang@gmail.com> wrote:
>>> Please correct me if I'm wrong. My understanding of aggregating the log
>> is
>>> the collect the generated from each monitored machine in real time. The
>>> collecting procedure is continuous like a data stream and never end.
>>> 
>>> I know how to use Hama to aggregate the logs batch by batch (e.g.
>> aggregate
>>> the logs incrementally each day), but I cannot immediately make up an
>> idea
>>> of using Hama to solve this problem in real time approach.
>>> 
>>> 
>>> 2014-03-04 19:32 GMT-05:00 Edward J. Yoon <edwardyoon@apache.org>:
>>> 
>>>> Aggregators of Graph package are doing similar wok. Monitoring and
>>>> Global communication, ..., etc.
>>>> 
>>>> 
>>>> 
>>>> On Tue, Mar 4, 2014 at 10:20 PM, Yexi Jiang <yexijiang@gmail.com>
>> wrote:
>>>>> I am very interested in this topic since my research area includes
>> event
>>>>> mining, but can BSP conducts the real time computing?
>>>>> 
>>>>> I once used the message queue based solution to collect the event
>> logs.
>>>>> 
>>>>> 
>>>>> 2014-03-04 1:54 GMT-05:00 Edward J. Yoon (JIRA) <jira@apache.org>:
>>>>> 
>>>>>> 
>>>>>>     [
>> https://issues.apache.org/jira/browse/HAMA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>>>> ]
>>>>>> 
>>>>>> Edward J. Yoon updated HAMA-883:
>>>>>> --------------------------------
>>>>>> 
>>>>>>    Summary: [Research Task] Massive log event aggregation in real
>> time
>>>>>> using Apache Hama  (was: [Research Task] Massive log data
>> aggregation in
>>>>>> real time using Apache Hama)
>>>>>> 
>>>>>>> [Research Task] Massive log event aggregation in real time using
>>>> Apache
>>>>>> Hama
>> ----------------------------------------------------------------------------
>>>>>>> 
>>>>>>>                Key: HAMA-883
>>>>>>>                URL:
>> https://issues.apache.org/jira/browse/HAMA-883
>>>>>>>            Project: Hama
>>>>>>>         Issue Type: Task
>>>>>>>           Reporter: Edward J. Yoon
>>>>>>> 
>>>>>>> BSP tasks can be used for aggregating log data streamed in real
>> time.
>>>>>> With this research task, we might able to platformization these kind
>> of
>>>>>> processing.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> This message was sent by Atlassian JIRA
>>>>>> (v6.2#6252)
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> ------
>>>>> Yexi Jiang,
>>>>> ECS 251,  yjian004@cs.fiu.edu
>>>>> School of Computer and Information Science,
>>>>> Florida International University
>>>>> Homepage: http://users.cis.fiu.edu/~yjian004/
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Edward J. Yoon (@eddieyoon)
>>>> Chief Executive Officer
>>>> DataSayer, Inc.
>>> 
>>> 
>>> 
>>> --
>>> ------
>>> Yexi Jiang,
>>> ECS 251,  yjian004@cs.fiu.edu
>>> School of Computer and Information Science,
>>> Florida International University
>>> Homepage: http://users.cis.fiu.edu/~yjian004/
> 
> 
> 
> -- 
> ------
> Yexi Jiang,
> ECS 251,  yjian004@cs.fiu.edu
> School of Computer and Information Science,
> Florida International University
> Homepage: http://users.cis.fiu.edu/~yjian004/

Mime
View raw message