Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 44EC9E4D7 for ; Sun, 3 Mar 2013 04:14:43 +0000 (UTC) Received: (qmail 85926 invoked by uid 500); 3 Mar 2013 04:14:38 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 85570 invoked by uid 500); 3 Mar 2013 04:14:37 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 85553 invoked by uid 99); 3 Mar 2013 04:14:36 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Mar 2013 04:14:36 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.214.182] (HELO mail-ob0-f182.google.com) (209.85.214.182) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Mar 2013 04:14:30 +0000 Received: by mail-ob0-f182.google.com with SMTP id va7so1306175obc.41 for ; Sat, 02 Mar 2013 20:14:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=j2UDeEyaDLIitNRDdSFw0OjjluhQa0pY4GwUMdc+GoQ=; b=ROUUwRXZuMUy+JUwRn/Z1SiPPioAkCUbf41mPjR368QL+0t9jsNp86/BSkWmg2GM2q xapOqNaKc3v82xFLNms0mtAw+Mskde8GaTx9+vHevJYPTzLxHc2fNDqOQGVRKshj98Re gIY3KSMkpKJ/vh/61f9Xdxa6XOW/lt+xivIf12VtJvv17EV49sapXlE7b++XVdvMCGhc 8CDdOL3gBF0tbNZC7rNLmpzWxwC8OsA0OOjCbVJbjuyWS5DWEamgwFqoZ6LhcQArj/+Z HgNx9qF+W20KNYFAJH7TBnk4rmSSiehb9OgR+A+x54R+0O48yjgIc7GjA7gC0b1zQ/+R U35Q== MIME-Version: 1.0 X-Received: by 10.182.151.9 with SMTP id um9mr12600814obb.89.1362284048123; Sat, 02 Mar 2013 20:14:08 -0800 (PST) Received: by 10.60.20.38 with HTTP; Sat, 2 Mar 2013 20:14:08 -0800 (PST) In-Reply-To: <6c68db78.ec5b.13d2a2e7616.Coremail.springring@126.com> References: <6c68db78.ec5b.13d2a2e7616.Coremail.springring@126.com> Date: Sun, 3 Mar 2013 08:14:08 +0400 Message-ID: Subject: Re: WholeFileInputFormat with streaming From: Fatih Haltas To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=f46d0444e925c59d8304d6fd782e X-Gm-Message-State: ALoCoQmyb09Z80v3qaN7hP3rKq/M3gIq23DZ6+4klYeLp4ev1+dkMWhPjsz7Uy2gZXCeh0QqwwUO X-Virus-Checked: Checked by ClamAV on apache.org --f46d0444e925c59d8304d6fd782e Content-Type: text/plain; charset=ISO-8859-9 Content-Transfer-Encoding: quoted-printable Hi Ring, Can you write output of "jar tf filename" command to see package organization of jar, you created? 2 Mart 2013 Cumartesi tarihinde springring adl=FD kullan=FDc=FD =FE=F6yle y= azd=FD: > Hi, > > I want to use: > hadoop jar /hadoop-streaming-0.20.2-cdh3u3.jar -inputformat > org.apache.hadoop.streaming.WholeFileInputFormat > > so, I download code from : > https://github.com/tomwhite/hadoop-book/tree/master/ch07/src/main/java > WholeFileInputFormat.java > WholeFil! eRecordRe ader.java > > and package the java file with : > package org.apache.hadoop.streaming; > > solution A: > copy WholeFileInputFormat.java, > WholeFileRecordReader.java > to > hadoop-0.20.2-cdh3u3/src/contrib/streaming/src/java/org/apache/hadoop/str= eaming/ > then > javac -classpath > /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/usr/lib/hadoop-0.20:/= usr/lib/hadoop-0.20/lib/* > -d ! WFInputFo rmatClassNew > hadoop-0.20.2-cdh3u3/src/contrib/streaming/src/java/org/apache/hadoop/str= eaming/*.java > > there is a lot of error > > solution B: > compile the java file WholeFileInputFormat.java, > WholeFileRecordReader.java > : > javac -classpath > /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/usr/lib/hadoop-0.20/*= :/usr/lib/hadoop-0.20/lib/* > -d WFInputFormatClass > copy /usr/l! ib/hadoop > /contrib/streaming/hadoop-streaming-0.20.2-cdh3u3.jar to > then : > jar uf hadoop-streaming-0.20.2-cdh3u3.jar > WFInputFormatClass/org/apache/hadoop/streaming/WholeFileRecordReader.clas= s > jar uf hadoop-streaming-0.20.2-cdh3u3.jar > WFInputFormatClass/org/apache/hadoop/streaming/WholeFileInputFormat.class > there is no error, but when I run: > hadoop jar /hadoop-streaming-0.20.2-cdh3u3.jar -inputformat > org.apache.hadoop.streaming.WholeFileInputFormat ....... > there is error: > -inputformat : class not found : > org.apache.hadoop.streaming.WholeFileInputFormat > > what's wrong with the two solution? or is there any new solution? > > thx. > > Ring > > > --f46d0444e925c59d8304d6fd782e Content-Type: text/html; charset=ISO-8859-9 Content-Transfer-Encoding: quoted-printable Hi Ring,

Can you write output of=A0"jar tf filename" command<= span>=A0to see package organization of jar, you created?=A0

2= Mart 2013 Cumartesi tarihinde springring adl=FD kullan=FDc=FD =FE=F6yle y= azd=FD:

Hi,

= =A0=A0=A0 I want to use:
hadoop jar <my dir>/hadoop-streaming-0.20= .2-cdh3u3.jar -inputformat org.apache.hadoop.streaming.WholeFileInputFormat=

=A0=A0=A0 so, I download code from :=A0=A0
https://github.com/tomwhite/hadoop-book/tree/master/ch07/src/main/java
WholeFileInputFormat.java
WholeFil! eRecordRe ader.java

=A0=A0=A0 and package the java file with :
package= org.apache.hadoop.streaming;

solution A:
copy Whole= FileInputFormat.java , WholeFileRecordReader.java=A0= to=A0 hadoop-0.20.2-cdh3u3/src/contrib/streaming/src/java/org/apache/hadoo= p/streaming/
then
javac -classpath /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.ja= r:/usr/lib/hadoop-0.20:/usr/lib/hadoop-0.20/lib/* -d ! WFInputFo rmatClassNew hadoop-0.20.2-cdh3u3/src/contrib/streaming/src/java/org/apach= e/hadoop/streaming/*.java

there is a lot of error

solution B:=
compile the java file=A0=A0 WholeFileInputFormat.java = , WholeFileRecordReader.java:
javac -classpath /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/usr/li= b/hadoop-0.20/*:/usr/lib/hadoop-0.20/lib/* -d WFInputFormatClass <two ja= va file>
copy /usr/l! ib/hadoop /contrib/streaming/hadoop-streaming-0.20.2-cdh3u3.jar=A0=A0 to <my dir&= gt;
then :
jar uf hadoop-streaming-0.20.2-cdh3u3.jar WFInputFormatCla= ss/org/apache/hadoop/streaming/WholeFileRecordReader.class
jar uf hadoop= -streaming-0.20.2-cdh3u3.jar WFInputFormatClass/org/apache/hadoop/streaming= /WholeFileInputFormat.class
there is no error, but when I run:
hadoop jar <my dir>/hadoop-stre= aming-0.20.2-cdh3u3.jar -inputformat org.apache.hadoop.streaming.WholeFileI= nputFormat .......
there is error:
-inputformat : class not found : o= rg.apache.hadoop.streaming.WholeFileInputFormat

what's wrong with the two solution? or is there any new solution?
thx.

Ring

--f46d0444e925c59d8304d6fd782e--