hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: troubles with HBase unit tests using MiniMRCluster on 0.23.1-SNAPSHOT
Date Wed, 11 Jan 2012 02:30:58 GMT
Hi Andrew,
 Thanks for filing MAPREDUCE-3654. Ill take a look. As for the
filenotfound issue
mvn -Dhadoop.profile=23 clean test
-Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce

on hbase 0.92. (latest svn  branch)  should be able to recreate this?
(Assuming your patch applied on MR-3654)

I can try that on my setup and see what the issue is.

thanks
mahadev

On Tue, Jan 10, 2012 at 6:11 PM, Andrew Purtell <apurtell@apache.org> wrote:
> Hi,
>
> I've been trying to test HBase 0.92 (prerelease) with 0.23.1-SNAPSHOT but
> have run into a couple of issues. Perhaps I'm doing something wrong.
>
> What I've done:
>
>   - Checked out Hadoop branch-0.23 and installed artifacts into the local
> maven cache:
>
>      $ cd /path/to/hadoop
>      $ mvn clean install -DskipTests
>
>   - Checked out HBase branch-0.92 and attempted to test under the maven
> profile for 0.23:
>
>      $ cd /path/to/hbase
>      $ mvn -Dhadoop.profile=23 clean test
> -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
>
> The first issue I encountered was that the MiniMRCluster would not
> initialize:
>
> 2012-01-10 18:01:02,489 INFO  [main] mapreduce.TestTableMapReduce(153):
> Started mrtest
> 2012-01-10 18:01:02,493 DEBUG [main] mapreduce.Cluster(93): Trying
> ClientProtocolProvider :
> org.apache.hadoop.mapred.LocalClientProtocolProvider
> 2012-01-10 18:01:02,494 INFO  [main] mapreduce.Cluster(116): Failed to use
> org.apache.hadoop.mapred.LocalClientProtocolProvider due to error: Invalid
> "mapreduce.jobtracker.address" configuration value for LocalJobRunner :
> "test"
> 2012-01-10 18:01:02,494 ERROR [main] security.UserGroupInformation(1160):
> PriviledgedActionException as:apurtell (auth:SIMPLE)
> cause:java.io.IOException: Cannot initialize Cluster. Please check your
> configuration for mapreduce.framework.name and the correspond server
> addresses.
> 2012-01-10 18:01:02,497 INFO  [main] hbase.HBaseTestingUtility(1173):
> Stopping mini mapreduce cluster...
> 2
>
> I filed MAPREDUCE-3654
> (https://issues.apache.org/jira/browse/MAPREDUCE-3654). There could be more
> going on here, but the patch on that issue let me move forward.
>
> Now I am seeing this:
>
> 2012-01-10 17:51:35,225 DEBUG [main] mapreduce.Cluster(106): Picked
> org.apache.hadoop.mapred.LocalClientProtocolProvider as the
> ClientProtocolProvider
> 2012-01-10 17:51:35,243 INFO  [main] mapreduce.TableOutputFormat(197):
> Created table instance for mrtest
> 2012-01-10 17:51:35,249 DEBUG [main] mapreduce.JobSubmitter(347):
> Configuring job job_local_0001 with
> file:/tmp/hadoop-apurtell/mapred/staging/apurtell-129099179/.staging/job_local_0001
> as the submit dir
> 2012-01-10 17:51:35,251 DEBUG [main] mapreduce.JobSubmitter(565): adding the
> following namenodes' delegation tokens:[hdfs://localhost.localdomain:54525]
> 2012-01-10 17:51:35,251 WARN  [main] mapreduce.JobSubmitter(139): Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2012-01-10 17:51:35,252 DEBUG [main] mapreduce.JobSubmitter(158): default
> FileSystem: file:///
> 2012-01-10 17:51:35,261 WARN  [main] mapreduce.JobSubmitter(241): No job jar
> file set.  User classes may not be found. See Job or Job#setJar(String).
> 2012-01-10 17:51:35,266 INFO  [main] mapreduce.JobSubmitter(388): Cleaning
> up the staging area
> file:/tmp/hadoop-apurtell/mapred/staging/apurtell-129099179/.staging/job_local_0001
> 2012-01-10 17:51:35,268 ERROR [main] security.UserGroupInformation(1160):
> PriviledgedActionException as:apurtell (auth:SIMPLE)
> cause:java.io.FileNotFoundException: File does not exist:
> /home/apurtell/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar
> 2012-01-10 17:51:35,273 INFO  [main] hbase.HBaseTestingUtility(1173):
> Stopping mini mapreduce cluster...
>
> Of course that FileNotFoundException is for a file that does indeed exist,
> in the local maven cache:
>
>     $ ls -l
> /home/apurtell/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar
>     -rw-rw-r-- 1 apurtell apurtell 764555 2012-01-05 18:56
> /home/apurtell/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar
>
> I set permissions on all directories from /home/apurtell to the jar to
> 'rwxr-xr-x' just to make sure there was not some kind of odd permissions
> problem.
>
> Any idea what is going on here?
>
> Best regards,
>
> - Andy
>
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Mime
View raw message