hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Illecker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HAMA-839) Support NullWritable in Hama Pipes
Date Thu, 09 Jan 2014 18:53:50 GMT

     [ https://issues.apache.org/jira/browse/HAMA-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Martin Illecker updated HAMA-839:
---------------------------------

    Description: 
h3. Support NullWritable in Hama Pipes
It should be possible to read, write and send NullWritables in Hama Pipes.

Therefore the *void* type is supported in C++ templates.
e.g., PiEstimator (InputKeyClass, InputValueClass and OutputKeyClass are NullWritables)
{code}
-class PiEstimatorBSP: public BSP<string,string,string,double,int> {
+class PiEstimatorBSP: public BSP<void,void,void,double,int> {
...
-  void cleanup(BSPContext<string,string,string,double,int>& context) {
+  void cleanup(BSPContext<void,void,void,double,int>& context) {
...
     double pi = 4.0 * total_hits / (msg_count * iterations_);
-    context.write("Estimated value of PI", pi);
+    context.write(pi);
     }
  }
{code}

This is affecting the following methods:

* Input
|{code}bool readNext(K1& key, V1& value){code}| method is available at compile time
if {{BSP<*K1!=void*, *V1!=void*, ?, ?, ?>}}|
|{code}bool readNext(K1& key){code}| method is available at compile time if {{BSP<*K1!=void*,
*V1==void*, ?, ?, ?>}}|
|{code}bool readNext(V1& value){code}| method is available at compile time if {{BSP<*K1==void*,
*V1!=void*, ?, ?, ?>}}|
| | method will *not* be available at compile time if {{BSP<*K1==void*, *V1==void*, ?,
?, ?>}}|

* Output
|{code}void write(const K2& key, const V2& value){code}| method is available at compile
time if {{BSP<?, ?, *K2!=void*, *V2!=void*, ?>}}|
|{code}void write(const K2& key){code}| method is available at compile time if {{BSP<?,
?, *K2!=void*, *V2==void*, ?>}}|
|{code}void write(const V2& value){code}| method is available at compile time if {{BSP<?,
?, *K2==void*, *V2!=void*, ?>}}|
| | method will *not* be available at compile time if {{BSP<?, ?, *K2==void*, *V2==void*,
?>}}|

* Message
|{code}void sendMessage(const string& peerName, const M& msg){code}| method will *not*
be available at compile time if {{BSP<?, ?, ?, ?, *M==void*>}}|
|{code}M getCurrentMessage(){code}| method will *not* be available at compile time if {{BSP<?,
?, ?, ?, *M==void*>}}|

Hama Pipes supports NullWritables in *SequenceFileInputFormat* and *SequenceFileOutputFormat*.

The same is valid for the *SequenceFile* methods:

* SequenceFile.Reader
|{code}bool sequenceFileReadNext<K, V>(int fileID, K& key, V& value){code}|
method is available at compile time if {{<*K!=void*, *V!=void*>}}|
|{code}bool sequenceFileReadNext<K>(int fileID, K& key){code}| method is available
at compile time if {{<*K!=void*>}}|
|{code}bool sequenceFileReadNext<V>(int fileID, K& key){code}| method is available
at compile time if {{<*V!=void*>}}|

* SequenceFile.Writer
|{code}bool sequenceFileAppend<K, V>(int fileID, const K& key, const V& value){code}|
method is available at compile time if {{<*K!=void*, *V!=void*>}}|
|{code}bool sequenceFileAppend <K>(int fileID, K& key){code}| method is available
at compile time if {{<*K!=void*>}}|
|{code}bool sequenceFileAppend <V>(int fileID, K& key){code}| method is available
at compile time if {{<*V!=void*>}}|


  was:
h3. Support NullWritable in Hama Pipes
It should be possible to read, write and send NullWritables in Hama Pipes.

Therefore the *void* type is supported in C++ templates.
e.g., PiEstimator (InputKeyClass, InputValueClass and OutputKeyClass are NullWritables)
{code}
-class PiEstimatorBSP: public BSP<string,string,string,double,int> {
+class PiEstimatorBSP: public BSP<void,void,void,double,int> {
...
-  void cleanup(BSPContext<string,string,string,double,int>& context) {
+  void cleanup(BSPContext<void,void,void,double,int>& context) {
...
     double pi = 4.0 * total_hits / (msg_count * iterations_);
-    context.write("Estimated value of PI", pi);
+    context.write(pi);
     }
  }
{code}

This is affecting the following methods:

* Input
|{code}bool readNext(K1& key, V1& value){code}| method is available at compile time
if {{BSP<*K1!=void*, *V1!=void*, ?, ?, ?>}}|
|{code}bool readNext(K1& key){code}| method is available at compile time if {{BSP<*K1!=void*,
*V1==void*, ?, ?, ?>}}|
|{code}bool readNext(V1& value){code}| method is available at compile time if {{BSP<*K1==void*,
*V1!=void*, ?, ?, ?>}}|
| | method will *not* be available at compile time if {{BSP<*K1==void*, *V1==void*, ?,
?, ?>}}|

* Output
|{code}void write(const K2& key, const V2& value){code}| method is available at compile
time if {{BSP<?, ?, *K2==void*, *V2==void*, ?>}}|
|{code}void write(const K2& key, const V2& value){code}| method is available at compile
time if {{BSP<?, ?, *K2==void*, *V2==void*, ?>}}|
|{code}void write(const K2& key, const V2& value){code}| method is available at compile
time if {{BSP<?, ?, *K2==void*, *V2==void*, ?>}}|
| | method will *not* be available at compile time if {{BSP<?, ?, *K2==void*, *V2==void*,
?>}}|

* Message
|{code}void sendMessage(const string& peerName, const M& msg){code}| method will *not*
be available at compile time if {{BSP<?, ?, ?, ?, *M==void*>}}|
|{code}M getCurrentMessage(){code}| method will *not* be available at compile time if {{BSP<?,
?, ?, ?, *M==void*>}}|

Hama Pipes supports NullWritables in *SequenceFileInputFormat* and *SequenceFileOutputFormat*.

The same is also valid for the *SequenceFile* methods:

* SequenceFile.Reader

bool sequenceFileReadNext(int fileID, K& key, V& value)

bool sequenceFileAppend(int fileID, const K& key, const V& value)

* SequenceFile.Writer


> Support NullWritable in Hama Pipes
> ----------------------------------
>
>                 Key: HAMA-839
>                 URL: https://issues.apache.org/jira/browse/HAMA-839
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp core, pipes
>    Affects Versions: 0.6.3
>            Reporter: Martin Illecker
>            Assignee: Martin Illecker
>            Priority: Minor
>              Labels: pipes
>             Fix For: 0.7.0
>
>         Attachments: HAMA-839_1.patch, HAMA-839_2.patch
>
>
> h3. Support NullWritable in Hama Pipes
> It should be possible to read, write and send NullWritables in Hama Pipes.
> Therefore the *void* type is supported in C++ templates.
> e.g., PiEstimator (InputKeyClass, InputValueClass and OutputKeyClass are NullWritables)
> {code}
> -class PiEstimatorBSP: public BSP<string,string,string,double,int> {
> +class PiEstimatorBSP: public BSP<void,void,void,double,int> {
> ...
> -  void cleanup(BSPContext<string,string,string,double,int>& context) {
> +  void cleanup(BSPContext<void,void,void,double,int>& context) {
> ...
>      double pi = 4.0 * total_hits / (msg_count * iterations_);
> -    context.write("Estimated value of PI", pi);
> +    context.write(pi);
>      }
>   }
> {code}
> This is affecting the following methods:
> * Input
> |{code}bool readNext(K1& key, V1& value){code}| method is available at compile
time if {{BSP<*K1!=void*, *V1!=void*, ?, ?, ?>}}|
> |{code}bool readNext(K1& key){code}| method is available at compile time if {{BSP<*K1!=void*,
*V1==void*, ?, ?, ?>}}|
> |{code}bool readNext(V1& value){code}| method is available at compile time if {{BSP<*K1==void*,
*V1!=void*, ?, ?, ?>}}|
> | | method will *not* be available at compile time if {{BSP<*K1==void*, *V1==void*,
?, ?, ?>}}|
> * Output
> |{code}void write(const K2& key, const V2& value){code}| method is available
at compile time if {{BSP<?, ?, *K2!=void*, *V2!=void*, ?>}}|
> |{code}void write(const K2& key){code}| method is available at compile time if {{BSP<?,
?, *K2!=void*, *V2==void*, ?>}}|
> |{code}void write(const V2& value){code}| method is available at compile time if
{{BSP<?, ?, *K2==void*, *V2!=void*, ?>}}|
> | | method will *not* be available at compile time if {{BSP<?, ?, *K2==void*, *V2==void*,
?>}}|
> * Message
> |{code}void sendMessage(const string& peerName, const M& msg){code}| method will
*not* be available at compile time if {{BSP<?, ?, ?, ?, *M==void*>}}|
> |{code}M getCurrentMessage(){code}| method will *not* be available at compile time if
{{BSP<?, ?, ?, ?, *M==void*>}}|
> Hama Pipes supports NullWritables in *SequenceFileInputFormat* and *SequenceFileOutputFormat*.
> The same is valid for the *SequenceFile* methods:
> * SequenceFile.Reader
> |{code}bool sequenceFileReadNext<K, V>(int fileID, K& key, V& value){code}|
method is available at compile time if {{<*K!=void*, *V!=void*>}}|
> |{code}bool sequenceFileReadNext<K>(int fileID, K& key){code}| method is available
at compile time if {{<*K!=void*>}}|
> |{code}bool sequenceFileReadNext<V>(int fileID, K& key){code}| method is available
at compile time if {{<*V!=void*>}}|
> * SequenceFile.Writer
> |{code}bool sequenceFileAppend<K, V>(int fileID, const K& key, const V&
value){code}| method is available at compile time if {{<*K!=void*, *V!=void*>}}|
> |{code}bool sequenceFileAppend <K>(int fileID, K& key){code}| method is available
at compile time if {{<*K!=void*>}}|
> |{code}bool sequenceFileAppend <V>(int fileID, K& key){code}| method is available
at compile time if {{<*V!=void*>}}|



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message