Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Subject: Re: C++ pipes on full (nonpseudo) cluster
From: Gianluigi Zanetti <gianluigi.zanetti@crs4.it>
Reply-To: gianluigi.zanetti@crs4.it
To: common-user@hadoop.apache.org
In-Reply-To: <919BE350-EA26-40B0-8651-FC9FE8B46605@keithwiley.com>
References: <AADFC164-F65B-4A0B-97D4-01BB1B4EB4C0@u.washington.edu>
	 <919BE350-EA26-40B0-8651-FC9FE8B46605@keithwiley.com>
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Organization: CRS4
Date: Tue, 30 Mar 2010 22:43:37 +0200
Message-Id: <1269981817.7630.41.camel@pflip>
Mime-Version: 1.0

Hello.
Did you try following the tutorial in 
http://wiki.apache.org/hadoop/C++WordCount ?

We use C++ pipes in production on a large cluster, and it works.

--gianluigi


On Tue, 2010-03-30 at 13:28 -0700, Keith Wiley wrote:
> No responses yet, although I admit it's only been a few hours.
> 
> As a follow-up, permit me to pose the following question:
> 
> Is it, in fact, impossible to run C++ pipes on a fully-distributed system (as opposed to a pseudo-distributed system)?  I haven't found any definitive clarification on this topic one way or the other.  The only statement that I found in the least bit illuminating is in the O'Reilly book (not official Hadoop documentation mind you), p.38, which states:
> 
> "To run a Pipes job, we need to run Hadoop in pseudo-distributed mode...Pipes doesn't run in standalone (local) mode, since it relies on Hadoop's distributed cache mechanism, which works only when HDFS is running."
> 
> The phrasing of those statements is a little unclear in that the distinction being made appears to be between standalone and pseudo-distributed mode, without any specific reference to fully-distributed mode.  Namely, the section that qualifies the need for pseudo-distributed mode (the need for HDFS) would obviously also apply to full distributed mode despite the lack of mention of fully distributed mode in the quoted section.  So can pipes run in fully distributed mode or not?
> 
> Bottom line, I can't get C++ pipes to work on a fully distributed cluster yet and I don't know if I am wasting my time, if this is a truly impossible effort or if it can be done and I simply haven't figured out how to do it yet.
> 
> Thanks for any help.
> 
> ________________________________________________________________________________
> Keith Wiley               kwiley@keithwiley.com               www.keithwiley.com
> 
> "The easy confidence with which I know another man's religion is folly teaches
> me to suspect that my own is also."
>   -- Mark Twain
> ________________________________________________________________________________
> 
> 
>