hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yexi Jiang <yexiji...@gmail.com>
Subject Re: [jira] [Updated] (HAMA-883) [Research Task] Massive log event aggregation in real time using Apache Hama
Date Wed, 05 Mar 2014 04:09:58 GMT
Yes, currently Hama does not support streaming input and streaming output.
 That's why currently it is not a natural choice for people with real time
computing needs.

Do we really need to make Hama to support the real time computing? In that
case, we need to compete with Storm...


2014-03-04 22:58 GMT-05:00 Chia-Hung Lin <clin4j@googlemail.com>:

> I used Twitter Storm previously. Storm is an excellent framework in
> real time processing.
>
> Considering Hama in real time tasks, the framework in my opinion need
> to decouple io from hdfs so that the source/ input is not restricted
> to just hdfs.
>
> On 5 March 2014 09:30, Yexi Jiang <yexijiang@gmail.com> wrote:
> > Please correct me if I'm wrong. My understanding of aggregating the log
> is
> > the collect the generated from each monitored machine in real time. The
> > collecting procedure is continuous like a data stream and never end.
> >
> > I know how to use Hama to aggregate the logs batch by batch (e.g.
> aggregate
> > the logs incrementally each day), but I cannot immediately make up an
> idea
> > of using Hama to solve this problem in real time approach.
> >
> >
> > 2014-03-04 19:32 GMT-05:00 Edward J. Yoon <edwardyoon@apache.org>:
> >
> >> Aggregators of Graph package are doing similar wok. Monitoring and
> >> Global communication, ..., etc.
> >>
> >>
> >>
> >> On Tue, Mar 4, 2014 at 10:20 PM, Yexi Jiang <yexijiang@gmail.com>
> wrote:
> >> > I am very interested in this topic since my research area includes
> event
> >> > mining, but can BSP conducts the real time computing?
> >> >
> >> > I once used the message queue based solution to collect the event
> logs.
> >> >
> >> >
> >> > 2014-03-04 1:54 GMT-05:00 Edward J. Yoon (JIRA) <jira@apache.org>:
> >> >
> >> >>
> >> >>      [
> >> >>
> >>
> https://issues.apache.org/jira/browse/HAMA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> ]
> >> >>
> >> >> Edward J. Yoon updated HAMA-883:
> >> >> --------------------------------
> >> >>
> >> >>     Summary: [Research Task] Massive log event aggregation in real
> time
> >> >> using Apache Hama  (was: [Research Task] Massive log data
> aggregation in
> >> >> real time using Apache Hama)
> >> >>
> >> >> > [Research Task] Massive log event aggregation in real time using
> >> Apache
> >> >> Hama
> >> >> >
> >> >>
> >>
> ----------------------------------------------------------------------------
> >> >> >
> >> >> >                 Key: HAMA-883
> >> >> >                 URL:
> https://issues.apache.org/jira/browse/HAMA-883
> >> >> >             Project: Hama
> >> >> >          Issue Type: Task
> >> >> >            Reporter: Edward J. Yoon
> >> >> >
> >> >> > BSP tasks can be used for aggregating log data streamed in real
> time.
> >> >> With this research task, we might able to platformization these kind
> of
> >> >> processing.
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> This message was sent by Atlassian JIRA
> >> >> (v6.2#6252)
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > ------
> >> > Yexi Jiang,
> >> > ECS 251,  yjian004@cs.fiu.edu
> >> > School of Computer and Information Science,
> >> > Florida International University
> >> > Homepage: http://users.cis.fiu.edu/~yjian004/
> >>
> >>
> >>
> >> --
> >> Edward J. Yoon (@eddieyoon)
> >> Chief Executive Officer
> >> DataSayer, Inc.
> >>
> >
> >
> >
> > --
> > ------
> > Yexi Jiang,
> > ECS 251,  yjian004@cs.fiu.edu
> > School of Computer and Information Science,
> > Florida International University
> > Homepage: http://users.cis.fiu.edu/~yjian004/
>



-- 
------
Yexi Jiang,
ECS 251,  yjian004@cs.fiu.edu
School of Computer and Information Science,
Florida International University
Homepage: http://users.cis.fiu.edu/~yjian004/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message