incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <>
Subject [jira] [Updated] (HAMA-439) Reorganize modules for YARN/MR-NextGen integration
Date Tue, 27 Sep 2011 14:21:12 GMT


Vinod Kumar Vavilapalli updated HAMA-439:


Yes, we can do it in place on trunk itself. Let us see how far we can go without breaking

Thomas, I know you are raring to go :) So I thought I'll attach the basic patch and scripts
so you can start hacking at a faster pace.

Attaching a patch, over which you need to run the attached script. The idea is that when we
eventually wish to commit the patch, we can change the mv commands in the script to svn mv.

The code still doesn't compile. We need to change some interactions. I'll postpone the details
till all the modules are in some shape:
 - hama-api: Strictly the api for writing the bsp programs
 - hama-client: Client code needed for submitting the jobs.
 - hama-server: New yarn client. Empty for now.
 - hama-common: Common stuff between the server and client code.

For some time, while we move around thing, I don't expect much changes to the patch but a
lotsa changes to the script.
> Reorganize modules for YARN/MR-NextGen integration
> --------------------------------------------------
>                 Key: HAMA-439
>                 URL:
>             Project: Hama
>          Issue Type: Sub-task
>    Affects Versions: 0.3.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Thomas Jungblut
>             Fix For: 0.4.0
>         Attachments: HAMA-349-20110927.txt,
> I'd like to get started on HamaV2 to work with YARN/MR-nextgen.
> Here's what I think should be the module structure:
>  - *api*: Code from HamaV1 related to APIs that will be moved to this new module. This
will mainly have the BSP APIs: _BSP_, _BSPPeerInterface_ and all the related records like
_BSPMessage_, _BSPMessageBundle etc.
>  - *core*: Current/HamaV1 server code. Will have the current _BSPMaster_/_GroomServer_
infrastructure. We should probably rename it to {{classic-server}} or something similar.
>  - *server*: New Hama server implementation to work with YARN. Will have _BSPApplicationMaster_,
_BSPTask_ and related code.
>  - *client*: Code from HamaV1 related to client that will be moved to this new module.
This will mainly have the client-submission classes/interfaces like _BSPJob_, _BSPJobClient_,
_JobStatus_ etc. It will be configurable to either run with the classic-server implementation
or with YARN.
> There will not be any package changes but only code-moves across modules, so we will
be backwards compatible.
> How does it look? Any thoughts/suggestions?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message