hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chia-Hung Lin <cli...@googlemail.com>
Subject Re: [jira] [Updated] (HAMA-883) [Research Task] Massive log event aggregation in real time using Apache Hama
Date Wed, 05 Mar 2014 04:42:45 GMT
Below is just my personal viewpoint. We can refactor bsp to be more
modularized so that people can choose if that fits their requirement.
Basically bsp is a generalized model, it may be good if we can create
a flexible framework.



On 5 March 2014 12:25, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Why not?
>
> Sent from my iPhone
>
>> On 2014. 3. 5., at 오후 1:09, Yexi Jiang <yexijiang@gmail.com> wrote:
>>
>> Yes, currently Hama does not support streaming input and streaming output.
>> That's why currently it is not a natural choice for people with real time
>> computing needs.
>>
>> Do we really need to make Hama to support the real time computing? In that
>> case, we need to compete with Storm...
>>
>>
>> 2014-03-04 22:58 GMT-05:00 Chia-Hung Lin <clin4j@googlemail.com>:
>>
>>> I used Twitter Storm previously. Storm is an excellent framework in
>>> real time processing.
>>>
>>> Considering Hama in real time tasks, the framework in my opinion need
>>> to decouple io from hdfs so that the source/ input is not restricted
>>> to just hdfs.
>>>
>>>> On 5 March 2014 09:30, Yexi Jiang <yexijiang@gmail.com> wrote:
>>>> Please correct me if I'm wrong. My understanding of aggregating the log
>>> is
>>>> the collect the generated from each monitored machine in real time. The
>>>> collecting procedure is continuous like a data stream and never end.
>>>>
>>>> I know how to use Hama to aggregate the logs batch by batch (e.g.
>>> aggregate
>>>> the logs incrementally each day), but I cannot immediately make up an
>>> idea
>>>> of using Hama to solve this problem in real time approach.
>>>>
>>>>
>>>> 2014-03-04 19:32 GMT-05:00 Edward J. Yoon <edwardyoon@apache.org>:
>>>>
>>>>> Aggregators of Graph package are doing similar wok. Monitoring and
>>>>> Global communication, ..., etc.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 4, 2014 at 10:20 PM, Yexi Jiang <yexijiang@gmail.com>
>>> wrote:
>>>>>> I am very interested in this topic since my research area includes
>>> event
>>>>>> mining, but can BSP conducts the real time computing?
>>>>>>
>>>>>> I once used the message queue based solution to collect the event
>>> logs.
>>>>>>
>>>>>>
>>>>>> 2014-03-04 1:54 GMT-05:00 Edward J. Yoon (JIRA) <jira@apache.org>:
>>>>>>
>>>>>>>
>>>>>>>     [
>>> https://issues.apache.org/jira/browse/HAMA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>>>>> ]
>>>>>>>
>>>>>>> Edward J. Yoon updated HAMA-883:
>>>>>>> --------------------------------
>>>>>>>
>>>>>>>    Summary: [Research Task] Massive log event aggregation in
real
>>> time
>>>>>>> using Apache Hama  (was: [Research Task] Massive log data
>>> aggregation in
>>>>>>> real time using Apache Hama)
>>>>>>>
>>>>>>>> [Research Task] Massive log event aggregation in real time
using
>>>>> Apache
>>>>>>> Hama
>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>                Key: HAMA-883
>>>>>>>>                URL:
>>> https://issues.apache.org/jira/browse/HAMA-883
>>>>>>>>            Project: Hama
>>>>>>>>         Issue Type: Task
>>>>>>>>           Reporter: Edward J. Yoon
>>>>>>>>
>>>>>>>> BSP tasks can be used for aggregating log data streamed in
real
>>> time.
>>>>>>> With this research task, we might able to platformization these
kind
>>> of
>>>>>>> processing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> This message was sent by Atlassian JIRA
>>>>>>> (v6.2#6252)
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ------
>>>>>> Yexi Jiang,
>>>>>> ECS 251,  yjian004@cs.fiu.edu
>>>>>> School of Computer and Information Science,
>>>>>> Florida International University
>>>>>> Homepage: http://users.cis.fiu.edu/~yjian004/
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Edward J. Yoon (@eddieyoon)
>>>>> Chief Executive Officer
>>>>> DataSayer, Inc.
>>>>
>>>>
>>>>
>>>> --
>>>> ------
>>>> Yexi Jiang,
>>>> ECS 251,  yjian004@cs.fiu.edu
>>>> School of Computer and Information Science,
>>>> Florida International University
>>>> Homepage: http://users.cis.fiu.edu/~yjian004/
>>
>>
>>
>> --
>> ------
>> Yexi Jiang,
>> ECS 251,  yjian004@cs.fiu.edu
>> School of Computer and Information Science,
>> Florida International University
>> Homepage: http://users.cis.fiu.edu/~yjian004/

Mime
View raw message