hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8611) Improve test coverage in pkg org.apache.hadoop.hbase.mapred
Date Mon, 17 Jun 2013 23:51:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686195#comment-13686195
] 

Nick Dimiduk commented on HBASE-8611:
-------------------------------------

Sorry Valdim, you are correct. I've reviewed your patch more closely and I'm now +1 for commit...

Except, {{TestTableMapReduceUtil#shoudBeValidMapReduceEvaluation}} fails when I run locally
on Hadoop2. Would you mind taking a look?

Here's the commands I ran and the surefire log output:

{noformat}
$ mvn -Dhadoop.profile=2.0 clean package -DskipTests
$ mvn -Dhadoop.profile=2.0 test -Dtest=org.apache.hadoop.hbase.mapred.TestDriver,org.apache.hadoop.hbase.mapred.TestGroupingTableMap,org.apache.hadoop.hbase.mapred.TestIdentityTableMap,org.apache.hadoop.hbase.mapred.TestRowCounter,org.apache.hadoop.hbase.mapred.TestSplitTable,org.apache.hadoop.hbase.mapred.TestTableMapReduceUtil
{noformat}

{noformat}
2013-06-17 16:41:03,900 INFO  [main] hbase.ResourceChecker(171): after: mapred.TestTableMapReduceUtil#shoudBeValidMapReduceWithPartitionerEvaluation
Thread=174 (was 174), OpenFileDescriptor=334 (was 336), MaxFileDescriptor=10240 (was 10240),
SystemLoadAverage=298 (was 315), ProcessCount=147 (was 147), AvailableMemoryMB=455 (was 458),
ConnectionCount=4 (was 4)
2013-06-17 16:41:03,949 INFO  [main] hbase.ResourceChecker(147): before: mapred.TestTableMapReduceUtil#shoudBeValidMapReduceEvaluation
Thread=174, OpenFileDescriptor=336, MaxFileDescriptor=10240, SystemLoadAverage=298, ProcessCount=147,
AvailableMemoryMB=456, ConnectionCount=4
2013-06-17 16:41:03,950 INFO  [main] mapred.TestTableMapReduceUtil(100): before
2013-06-17 16:41:03,950 INFO  [main] mapred.TestTableMapReduceUtil(102): before done
2013-06-17 16:41:03,952 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.zookeeper.ZooKeeper,
using jar /Users/ndimiduk/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar
2013-06-17 16:41:03,953 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class com.google.common.base.Function,
using jar /Users/ndimiduk/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar
2013-06-17 16:41:03,954 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class com.google.protobuf.Message,
using jar /Users/ndimiduk/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar
2013-06-17 16:41:03,954 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:03,954 DEBUG [main] mapreduce.TableMapReduceUtil(692): org.apache.hadoop.util.JarFinder
found.
2013-06-17 16:41:04,041 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.io.ImmutableBytesWritable,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-2053724549545712152.jar
2013-06-17 16:41:04,042 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:04,042 DEBUG [main] mapreduce.TableMapReduceUtil(692): org.apache.hadoop.util.JarFinder
found.
2013-06-17 16:41:04,262 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.client.Put,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-4313891093512368515.jar
2013-06-17 16:41:04,263 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.io.LongWritable,
using jar /Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-common/2.0.2-alpha/hadoop-common-2.0.2-alpha.jar
2013-06-17 16:41:04,264 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.io.Text,
using jar /Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-common/2.0.2-alpha/hadoop-common-2.0.2-alpha.jar
2013-06-17 16:41:04,265 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.mapred.lib.HashPartitioner,
using jar /Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.2-alpha/hadoop-mapreduce-client-core-2.0.2-alpha.jar
2013-06-17 16:41:04,266 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:04,266 DEBUG [main] mapreduce.TableMapReduceUtil(692): org.apache.hadoop.util.JarFinder
found.
2013-06-17 16:41:05,406 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.mapred.TableInputFormat,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-5943365653915313487.jar
2013-06-17 16:41:05,406 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.mapred.TextOutputFormat,
using jar /Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.2-alpha/hadoop-mapreduce-client-core-2.0.2-alpha.jar
2013-06-17 16:41:05,407 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.zookeeper.ZooKeeper,
using jar /Users/ndimiduk/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar
2013-06-17 16:41:05,408 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class com.google.common.base.Function,
using jar /Users/ndimiduk/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar
2013-06-17 16:41:05,409 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class com.google.protobuf.Message,
using jar /Users/ndimiduk/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar
2013-06-17 16:41:05,409 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:05,409 DEBUG [main] mapreduce.TableMapReduceUtil(692): org.apache.hadoop.util.JarFinder
found.
2013-06-17 16:41:05,488 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.io.ImmutableBytesWritable,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
2013-06-17 16:41:05,488 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:05,488 DEBUG [main] mapreduce.TableMapReduceUtil(692): org.apache.hadoop.util.JarFinder
found.
2013-06-17 16:41:05,691 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.client.Put,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-2651672527795445135.jar
2013-06-17 16:41:05,692 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.io.ImmutableBytesWritable,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
2013-06-17 16:41:05,692 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.client.Put,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-2651672527795445135.jar
2013-06-17 16:41:05,693 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.mapred.lib.HashPartitioner,
using jar /Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.2-alpha/hadoop-mapreduce-client-core-2.0.2-alpha.jar
2013-06-17 16:41:05,694 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:05,694 DEBUG [main] mapreduce.TableMapReduceUtil(692): org.apache.hadoop.util.JarFinder
found.
2013-06-17 16:41:06,864 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.mapred.TableInputFormat,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-1149375339751659803.jar
2013-06-17 16:41:06,864 DEBUG [main] mapreduce.TableMapReduceUtil(614): For class org.apache.hadoop.hbase.mapred.TableOutputFormat,
using jar /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-1149375339751659803.jar
2013-06-17 16:41:06,895 WARN  [main] mapreduce.JobSubmitter(138): Hadoop command-line option
parsing not performed. Implement the Tool interface and execute your application with ToolRunner
to remedy this.
2013-06-17 16:41:06,950 WARN  [main] mapreduce.JobSubmitter(247): No job jar file set.  User
classes may not be found. See Job or Job#setJar(String).
2013-06-17 16:41:06,953 ERROR [main] security.UserGroupInformation(1370): PriviledgedActionException
as:ndimiduk (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist: /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
2013-06-17 16:41:06,953 ERROR [main] security.UserGroupInformation(1370): PriviledgedActionException
as:ndimiduk (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist: /Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
{noformat}

Interestingly, {{hadoop-7878386878750133423.jar}} does exist on my filesystem after the test
execution terminates.
                
> Improve test coverage in pkg org.apache.hadoop.hbase.mapred
> -----------------------------------------------------------
>
>                 Key: HBASE-8611
>                 URL: https://issues.apache.org/jira/browse/HBASE-8611
>             Project: HBase
>          Issue Type: Test
>            Reporter: Vadim Bondarev
>            Assignee: Vadim Bondarev
>         Attachments: HBASE-8611-0.94-a.patch, HBASE-8611-0.94-c.patch, HBASE-8611-trunk-a.patch,
HBASE-8611-trunk-b.patch, HBASE-8611-trunk-c.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message