hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Schulze, Dean (Contractor)" <Dean_Schu...@cable.comcast.com>
Subject RE: Unit test can't resolve org.apache.hadoop.hbase.client.Table
Date Wed, 15 Apr 2015 16:04:52 GMT
Thanks for being so patient Ted.

Now I'm getting the error below after adding the two dependencies below.  This error happens
when the call to HBaseTestingUtility.startMiniCluster() is made:

java.lang.NoSuchMethodError: org.apache.hadoop.fs.FSOutputSummer.<init>(Ljava/util/zip/Checksum;II)V


		<dependency>
			<groupId>org.apache.hadoop</groupId>
			<artifactId>hadoop-minicluster</artifactId>
			<version>2.5.1</version>
		</dependency>
 	    <dependency>
			<groupId>org.apache.hadoop</groupId>
      		<artifactId>hadoop-common</artifactId>
      		<version>2.5.1</version>
      		<type>test-jar</type>
    	</dependency>

In my project the FSOutputSummer class is from the cdh 5.3.0 jar files:

$ jar tvf org/apache/hadoop/hadoop-common/2.5.0-cdh5.3.0/hadoop-common-2.5.0-cdh5.3.0.jar
| grep FSOutputSummer
  3886 Tue Dec 16 19:04:14 MST 2014 org/apache/hadoop/fs/ChecksumFs$ChecksumFSOutputSummer.class
  3977 Tue Dec 16 19:04:14 MST 2014 org/apache/hadoop/fs/FSOutputSummer.class
  2901 Tue Dec 16 19:04:12 MST 2014 org/apache/hadoop/fs/ChecksumFileSystem$ChecksumFSOutputSummer.class

We use CDH so maybe there's an incompatibility between the Apache Hadoop/HBase jar files and
the CDH jar files when it comes to using a MiniCluster.

Any thoughts?


-----Original Message-----
From: Ted Yu [mailto:yuzhihong@gmail.com] 
Sent: Tuesday, April 14, 2015 3:49 PM
To: user@hbase.apache.org
Subject: Re: Unit test can't resolve org.apache.hadoop.hbase.client.Table

$ jar tvf
~/.m2/repository/org/apache/hadoop/hadoop-common/2.5.1/hadoop-common-2.5.1-tests.jar
| grep StaticMapping
  3858 Fri Sep 05 16:05:30 PDT 2014
org/apache/hadoop/net/StaticMapping.class
  6368 Fri Sep 05 16:05:30 PDT 2014
org/apache/hadoop/net/TestStaticMapping.class

Here is snippet of dependency tree:

[INFO] +- org.apache.hadoop:hadoop-minicluster:jar:2.5.1:test
[INFO] |  +- org.apache.hadoop:hadoop-common:test-jar:tests:2.5.1:test

Here is snippet from pom.xml:

          <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-minicluster</artifactId>
            <version>${hadoop-two.version}</version>
Cheers

On Tue, Apr 14, 2015 at 2:28 PM, Schulze, Dean (Contractor) < Dean_Schulze@cable.comcast.com>
wrote:

> So I added these two dependencies and now I get this error.
>
> Are the transitive dependencies for Hadoop and hbase not mapped properly?
>
>
> java.lang.NoClassDefFoundError: org/apache/hadoop/net/StaticMapping
>
>         <dependency>
>             <groupId>org.apache.hadoop</groupId>
>             <artifactId>hadoop-hdfs</artifactId>
>             <version>2.5.1</version>
>         </dependency>
>
>                 <dependency>
>             <groupId>org.apache.hadoop</groupId>
>             <artifactId>hadoop-hdfs</artifactId>
>             <version>2.5.1</version>
>             <type>test-jar</type>
>             <scope>test</scope>
>         </dependency>
>
>
>
> -----Original Message-----
> From: Ted Yu [mailto:yuzhihong@gmail.com]
> Sent: Tuesday, April 14, 2015 3:10 PM
> To: user@hbase.apache.org
> Subject: Re: Unit test can't resolve 
> org.apache.hadoop.hbase.client.Table
>
> MiniDFSCluster is in this jar:
>
> jar tvf
>
> ~/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1
> -tests.jar
> | grep MiniDFSCluster
>   1261 Fri Sep 05 16:08:24 PDT 2014
> org/apache/hadoop/hdfs/MiniDFSCluster$1.class
>   7431 Fri Sep 05 16:08:24 PDT 2014
> org/apache/hadoop/hdfs/MiniDFSCluster$Builder.class
>   1498 Fri Sep 05 16:08:24 PDT 2014
> org/apache/hadoop/hdfs/MiniDFSCluster$DataNodeProperties.class
>   1396 Fri Sep 05 16:08:24 PDT 2014
> org/apache/hadoop/hdfs/MiniDFSCluster$NameNodeInfo.class
>  57246 Fri Sep 05 16:08:24 PDT 2014
> org/apache/hadoop/hdfs/MiniDFSCluster.class
>
> Take a look at the following in pom.xml (around line 1835):
>
>           <dependency>
>             <groupId>org.apache.hadoop</groupId>
>             <artifactId>hadoop-hdfs</artifactId>
>             <version>${hadoop-two.version}</version>
>             <type>test-jar</type>
>             <scope>test</scope>
>
> Cheers
>
> On Tue, Apr 14, 2015 at 1:52 PM, Schulze, Dean (Contractor) < 
> Dean_Schulze@cable.comcast.com> wrote:
>
> > That solves one problem, but now I get this:
> >
> > java.lang.NoClassDefFoundError: 
> > org/apache/hadoop/hdfs/MiniDFSCluster
> >
> > That class is in project in mysql-connector-java-5.1.12.jar, which 
> > doesn't seem right for an in-memory test of HBase.
> >
> > Do I need to add a test-jar or something else in the test scope?
> >
> >
> > -----Original Message-----
> > From: Ted Yu [mailto:yuzhihong@gmail.com]
> > Sent: Tuesday, April 14, 2015 2:26 PM
> > To: user@hbase.apache.org
> > Subject: Re: Unit test can't resolve 
> > org.apache.hadoop.hbase.client.Table
> >
> > Please add dependency on hbase-client module where Table resides.
> >
> > Cheers
> >
> > On Tue, Apr 14, 2015 at 1:18 PM, Schulze, Dean (Contractor) < 
> > Dean_Schulze@cable.comcast.com> wrote:
> >
> > > I'm using HBaseTestingUtility in a Junit test.  It compiles, but 
> > > when I run the test (either from the command line or from Eclipse) 
> > > it gives the error below:
> > >
> > > java.lang.Error: Unresolved compilation problem:
> > >                 The type org.apache.hadoop.hbase.client.Table 
> > > cannot be resolved. It is indirectly referenced from required 
> > > .class files
> > >
> > >
> > > Does this indicate that the transitive dependencies are incorrect?
> > > Here are my mvn dependency and code:
> > >
> > >                 <dependency>
> > >                                 <groupId>org.apache.hbase</groupId>
> > >                                 <artifactId>hbase-server</artifactId>
> > >                                 <version>1.0.0</version>
> > >                 </dependency>
> > >                     <dependency>
> > >
> > > <groupId>org.apache.hbase</groupId>
> > >                                 <artifactId>hbase-server</artifactId>
> > >                                 <version>1.0.0</version>
> > >                                 <type>test-jar</type>
> > >                 </dependency>
> > >
> > >
> > > private static HBaseTestingUtility utility;
> > >
> > >                 @Before
> > >                 public void setUp() throws Exception {
> > >                                 utility = new HBaseTestingUtility();
> > >                                 utility.startMiniCluster();
> > >                 }
> > >
> > >                 @Test
> > >                 public void test() {
> > >
> > >                                 try {
> > >                                                 HTableInterface 
> > > table = utility.createTable(Bytes.toBytes("TestTable"), TestCF);
> > >
> > >                                                 ....
> > >
> > >
> >
>
Mime
View raw message