Return-Path: 
 <user-return-428-apmail-hadoop-hdfs-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 145C6D8E2
	for <apmail-hadoop-hdfs-user-archive@minotaur.apache.org>;
 Wed, 15 Aug 2012 07:42:34 +0000 (UTC)
Received: (qmail 6753 invoked by uid 500); 15 Aug 2012 07:42:29 -0000
Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org
Received: (qmail 6630 invoked by uid 500); 15 Aug 2012 07:42:27 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 6615 invoked by uid 99); 15 Aug 2012 07:42:27 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Aug 2012 07:42:27 +0000
X-ASF-Spam-Status: No, hits=-0.7 required=5.0
	tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of haavard.kongsgaard@gmail.com
 designates 209.85.212.182 as permitted sender)
Received: from [209.85.212.182] (HELO mail-wi0-f182.google.com)
 (209.85.212.182)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Aug 2012 07:42:23 +0000
Received: by wibhq12 with SMTP id hq12so992495wib.11
        for <user@hadoop.apache.org>; Wed, 15 Aug 2012 00:42:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        bh=mzc1acQxExiG6munX/D9zfwYT55f0ahuE9eHYouTSkE=;
        b=tYyhMNUJ4LUV367dD+NbjFhppl5CJGMqHjJAVLXqpPBk++5oxcDDLIYizjBuadlAFh
         tVJHDSUqml+W24KB6qoLGv9XOZoiTuIYIe7Jb0o7hlqvvsj4RNaOvopgFerH+W/rTWz+
         Y7PzgcqoGcCaaObKAom4R11W0v08jQ3tUvLgzpEEuac/LzlLfNSIRe/bL5guP2OLANQ1
         igZZ7Y6OyAZlV3BWxzrlZ9rHOGRPyVeLflG6tHUHqC0WsGW9Bx+5mp4bYXR+0x9TWzlE
         oWDyYLXp/MfLcrbadvKdWvjLbceiz5rLXKG3KMNGdWqmVVz5I0Gw4jsDUVvkhcu6qJG8
         JlmQ==
MIME-Version: 1.0
Received: by 10.180.83.106 with SMTP id p10mr34803847wiy.21.1345016521984;
 Wed, 15 Aug 2012 00:42:01 -0700 (PDT)
Received: by 10.194.40.7 with HTTP; Wed, 15 Aug 2012 00:42:01 -0700 (PDT)
In-Reply-To: 
 <CAOcnVr0Wjdti+-E0BS8a0Qe=haZ-R92zDLTMoz0sC3qz_Moi7Q@mail.gmail.com>
References: 
 <CAKH910_jF41+YWcDFqWSqm1T9yPyDOnh9uKsys2ZDdebecaZ+w@mail.gmail.com>
	<CAOcnVr0Wjdti+-E0BS8a0Qe=haZ-R92zDLTMoz0sC3qz_Moi7Q@mail.gmail.com>
Date: Wed, 15 Aug 2012 09:42:01 +0200
Message-ID: 
 <CAKH910-2-v0ro+nXsUjNgwrF=k8H9id8S1YC7bSBgiOo_KZ2Nw@mail.gmail.com>
Subject: Re: pipes(pydoop) and hbase classpath
From: =?ISO-8859-1?Q?H=E5vard_Wahl_Kongsg=E5rd?=
 <haavard.kongsgaard@gmail.com>
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Virus-Checked: Checked by ClamAV on apache.org

Hi, my job config is

<property>
<name>mapred.input.format.class</name>
<value>org.apache.hadoop.hbase.mapred.TableInputFormat</value>
</property>

<property>
  <name>hadoop.pipes.java.recordreader</name>
  <value>true</value>
</property>


Exception in thread "main" java.lang.RuntimeException: Error in
configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtil=
s.java:93)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.j=
ava:64)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUti=
ls.java:117)
        at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:596=
)
        at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java=
:977)
        at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:96=
9)
        at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170=
)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupIn=
formation.java:1177)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.j=
ava:833)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1248)
        at org.apache.hadoop.mapred.pipes.Submitter.runJob(Submitter.java:2=
48)
        at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:479)
        at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:494=
)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessor=
Impl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod=
AccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtil=
s.java:88)
        ... 17 more
Caused by: java.lang.NullPointerException
        at org.apache.hadoop.hbase.mapred.TableInputFormat.configure(TableI=
nputFormat.java:51)


should I included the col names? according to the api it's deprecated?
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapred/TableInputFo=
rmat.html


-H=E5vard


On Tue, Aug 14, 2012 at 11:17 PM, Harsh J <harsh@cloudera.com> wrote:
> Hi,
>
> Per:
>
>> org.apache.hadoop.hbase.mapreduce.TableInputFormat not
> org.apache.hadoop.mapred.InputFormat
>
> Pydoop seems to be expecting you to pass it an old API class for
> InputFormat/etc. but you've passed in the newer class. I am unsure
> what part of your code exactly may be at fault since I do not have
> access to it, but you probably want to use the deprecated
> org.apache.hadoop.hbase.mapred.* package classes such as
> org.apache.hadoop.hbase.mapred.TableInputFormat, and not the
> org.apache.hadoop.hbase.mapreduce.* classes, as you are using at the
> moment.
>
> HTH!
>
> On Wed, Aug 15, 2012 at 2:39 AM, H=E5vard Wahl Kongsg=E5rd
> <haavard.kongsgaard@gmail.com> wrote:
>> Hi, I'am trying to read hbase key-values with pipes(pydoop). As hadoop
>> is unable to find the hbase jar files. I get
>>
>> Exception in thread "main" java.lang.RuntimeException:
>> java.lang.RuntimeException: class
>> org.apache.hadoop.hbase.mapreduce.TableInputFormat not
>> org.apache.hadoop.mapred.InputFormat
>>
>> have added export
>> HADOOP_CLASSPATH=3D/usr/lib/hbase/hbase-0.90.6-cdh3u4.jar to my
>> hadoop-env.sh
>>
>> According to the doc from cloudera,
>> https://ccp.cloudera.com/display/CDHDOC/HBase+Installation#HBaseInstalla=
tion-UsingMapReducewithHBase
>> TableMapReduceUtil.addDependencyJars(job); can be used as an
>> alternative. But is that possible with pipes?
>>
>> -H=E5vard
>
>
>
> --
> Harsh J



--=20
H=E5vard Wahl Kongsg=E5rd
Faculty of Medicine &
Department of Mathematical Sciences
NTNU

http://havard.security-review.net/