hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: writing files to HDFS (from c++/pipes)
Date Mon, 07 Dec 2009 23:30:00 GMT

On Dec 7, 2009, at 10:05 AM, horson wrote:

> i want to write a file to hdfs, using hadoop pipes. can anyone tell  
> me how
> to do that?

You either use a Java OutputFormat, which is the easiest, or you use  
libhdfs to write to HDFS from C++.

> i looked at the hadoop pipes source and it looked very restricted,  
> can i do everything in
> hadoop pipes that possible in java?

No, not everything is supported. It does support record readers,  
mappers, combiners, reducers, record writers, and counters from C++.  
It also provides the entire job configuration as a string->string map.

-- Owen

View raw message