hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Illecker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HAMA-619) Hama Pipes
Date Thu, 16 Aug 2012 09:15:38 GMT

     [ https://issues.apache.org/jira/browse/HAMA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Martin Illecker updated HAMA-619:
---------------------------------

    Description: 
*Adapting Hadoop Pipes to Hama for C++ integration.*

Today I finished the first version of Hama Pipes. :-)
I have added the package *org.apache.hama.pipes* which contains all pipes specific code. The
C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are also
some minor modifications in other packages but you can checkout the full Hama source here:
[https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]

The working *revision is 32*, for the changelog see [here|http://hadoop.illecker.at/?page_id=6].

Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]


Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP) 

*{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by
this method are not guaranteed to be received in a sent order.

*{{virtual const string& getCurrentMessage() = 0;}}*
bq. Returns a message from the peer's received messages queue (a FIFO).

*{{virtual int getNumCurrentMessages() = 0;}}*
bq. Returns the number of messages in the peer's received messages queue.

*{{virtual void sync() = 0;}}*
{quote}
Barrier Synchronization. 
Sends all the messages in the outgoing message queues to the corresponding remote peers.
{quote}

*{{virtual long getSuperstepCount() = 0;}}*
bq. Returns the count of current super-step
     
*{{virtual const string& getPeerName() = 0;}}*
bq. Returns the name of this peer in the format "hostname:port".
    
*{{virtual const string& getPeerName(int index) = 0;}}*
bq. Returns the name of n-th peer from sorted array by name.
    
*{{virtual int getPeerIndex() = 0;}}*
bq. Returns the index of this peer from sorted array by name.

*{{virtual vector<string> getAllPeerNames() = 0;}}*
bq. Returns the names of all the peers executing tasks from the same job (including this peer).
    
*{{virtual int getNumPeers() = 0;}}*
bq. Returns the number of peers
    
*{{virtual void clear() = 0;}}*
bq. Clears all queues entries.
    
*{{virtual void write(const string& key, const string& value) = 0;}}*
bq. Writes a key/value pair to the output collector
    
*{{virtual bool readNext(string& key, string& value) = 0;}}*
bq. Deserializes the next input key value into the given objects;

*{{virtual void reopenInput() = 0;}}*
bq. Closes the input and opens it right away, so that the file pointer is at the beginning
again.


  was:
*Adapting Hadoop Pipes to Hama for C++ integration.*

The SVN source is available here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/
Hama Pipes: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/
Hama Pipes C++: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/


Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP) 

*{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by
this method are not guaranteed to be received in a sent order.

*{{virtual const string& getCurrentMessage() = 0;}}*
bq. Returns a message from the peer's received messages queue (a FIFO).

*{{virtual int getNumCurrentMessages() = 0;}}*
bq. Returns the number of messages in the peer's received messages queue.

*{{virtual void sync() = 0;}}*
{quote}
Barrier Synchronization. 
Sends all the messages in the outgoing message queues to the corresponding remote peers.
{quote}

*{{virtual long getSuperstepCount() = 0;}}*
bq. Returns the count of current super-step
     
*{{virtual const string& getPeerName() = 0;}}*
bq. Returns the name of this peer in the format "hostname:port".
    
*{{virtual const string& getPeerName(int index) = 0;}}*
bq. Returns the name of n-th peer from sorted array by name.
    
*{{virtual int getPeerIndex() = 0;}}*
bq. Returns the index of this peer from sorted array by name.

*{{virtual vector<string> getAllPeerNames() = 0;}}*
bq. Returns the names of all the peers executing tasks from the same job (including this peer).
    
*{{virtual int getNumPeers() = 0;}}*
bq. Returns the number of peers
    
*{{virtual void clear() = 0;}}*
bq. Clears all queues entries.
    
*{{virtual void write(const string& key, const string& value) = 0;}}*
bq. Writes a key/value pair to the output collector
    
*{{virtual bool readNext(string& key, string& value) = 0;}}*
bq. Deserializes the next input key value into the given objects;

*{{virtual void reopenInput() = 0;}}*
bq. Closes the input and opens it right away, so that the file pointer is at the beginning
again.


    
> Hama Pipes
> ----------
>
>                 Key: HAMA-619
>                 URL: https://issues.apache.org/jira/browse/HAMA-619
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp core, messaging
>    Affects Versions: 0.6.0
>            Reporter: Martin Illecker
>            Assignee: Martin Illecker
>
> *Adapting Hadoop Pipes to Hama for C++ integration.*
> Today I finished the first version of Hama Pipes. :-)
> I have added the package *org.apache.hama.pipes* which contains all pipes specific code.
The C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are
also some minor modifications in other packages but you can checkout the full Hama source
here:
> [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]
> The working *revision is 32*, for the changelog see [here|http://hadoop.illecker.at/?page_id=6].
> Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
> Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]
> Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

> *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
> bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent
by this method are not guaranteed to be received in a sent order.
> *{{virtual const string& getCurrentMessage() = 0;}}*
> bq. Returns a message from the peer's received messages queue (a FIFO).
> *{{virtual int getNumCurrentMessages() = 0;}}*
> bq. Returns the number of messages in the peer's received messages queue.
> *{{virtual void sync() = 0;}}*
> {quote}
> Barrier Synchronization. 
> Sends all the messages in the outgoing message queues to the corresponding remote peers.
> {quote}
> *{{virtual long getSuperstepCount() = 0;}}*
> bq. Returns the count of current super-step
>      
> *{{virtual const string& getPeerName() = 0;}}*
> bq. Returns the name of this peer in the format "hostname:port".
>     
> *{{virtual const string& getPeerName(int index) = 0;}}*
> bq. Returns the name of n-th peer from sorted array by name.
>     
> *{{virtual int getPeerIndex() = 0;}}*
> bq. Returns the index of this peer from sorted array by name.
> *{{virtual vector<string> getAllPeerNames() = 0;}}*
> bq. Returns the names of all the peers executing tasks from the same job (including this
peer).
>     
> *{{virtual int getNumPeers() = 0;}}*
> bq. Returns the number of peers
>     
> *{{virtual void clear() = 0;}}*
> bq. Clears all queues entries.
>     
> *{{virtual void write(const string& key, const string& value) = 0;}}*
> bq. Writes a key/value pair to the output collector
>     
> *{{virtual bool readNext(string& key, string& value) = 0;}}*
> bq. Deserializes the next input key value into the given objects;
> *{{virtual void reopenInput() = 0;}}*
> bq. Closes the input and opens it right away, so that the file pointer is at the beginning
again.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message