incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: Summary of problems with HAMA-413 and Discussion
Date Sat, 27 Aug 2011 21:45:09 GMT
>
> 1. Barrier Synchronizations are not working well (with a 'bench' example).


We should definitely talk about alternatives to ZooKeepers barrier sync.
I'm always +1 for homebrew code which always works, instead of not working
framework stuff.
Maybe we should ask on the ZK mailing list are what the guys thinking about
why it is not working properly. Assuming that we coded correct, AFAIK we
just took the example code right?:)


> 3. Graph examples are not working.
>

Like already mentioned in some other threads, I don't see this working
without framework support. [1]
Miklos pointed me to the goldenorb sources[2], they have an I/O system which
is built on top of Hadoop's RecordReader and Inputformats. He mentioned that
he wanted to extend HAMA-409 with this.
What do you think?

[1] Another option would be the dynamic partition assignment with Zookeeper,
Miklos used in HAMA-409.
[2] https://github.com/raveldata/goldenorb

2011/8/26 Edward J. Yoon <edwardyoon@apache.org>

> Okay.
>
> Sent from my iPhone
>
> On 2011. 8. 26., at 오후 2:49, "ChiaHung Lin" <chl501@nuk.edu.tw> wrote:
>
> > The latest patch (HAMA_NEW.patch) for HAMA-413 seems still using bsp peer
> to report its status back to master.
> >
> > +        umbilical.updateTaskStatusAndReport(taskid);
> >
> > +  public void updateTaskStatusAndReport(TaskAttemptID taskid) {
> > ...
> > +    doReport(taskStatus);
> > +  }
> >
> > Is there any chance to revert back using a version that reports task
> status by GroomServer, so we can discuss based on that version? Just to
> ensure that the following issues are not the result derived from the code
> changed above.
> >
> > -----Original message-----
> > From:Edward J. Yoon <edwardyoon@apache.org>
> > To:hama-dev@incubator.apache.org
> > Date:Thu, 25 Aug 2011 19:43:48 +0900
> > Subject:Summary of problems with HAMA-413 and Discussion
> >
> > Today, I tested all Hama examples on my cluster of 32 nodes, with 96
> > tasks. Pi and Serialized Printing examples were working fine but
> >
> > 1. Barrier Synchronizations are not working well (with a 'bench'
> example).
> > 2. When an unexpected shutdown occurs, ZK nodes (which created by each
> > BSPPeer) will not be deleted. There's no way to clean them up before
> > reboot the server.
> > 3. Graph examples are not working.
> > 4. Too many reporting times between Groom and Master.
> > 5. And, there are many code issues that can be improved.
> >
> > 1, and 2 issues are already reported (See HAMA-387, HAMA-407). Some of
> > 3, 4, and 5 issues are already started by ChiaHung Lin.
> >
> > All issues around this should be fixed in HAMA-413? or, Should we just
> > commit HAMA-413?
> >
> > Thanks.
> > --
> > Best Regards, Edward J. Yoon
> > @eddieyoon
> >
> >
> > --
> > ChiaHung Lin
> > Department of Information Management
> > National University of Kaohsiung
> > Taiwan
>



-- 
Thomas Jungblut
Berlin

mobile: 0170-3081070

business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message