accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4467) Random Walk broken because of unmet dependency on commons-math
Date Wed, 21 Sep 2016 20:11:20 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511021#comment-15511021
] 

Hadoop QA commented on ACCUMULO-4467:
-------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s {color} | {color:red}
ACCUMULO-4467 does not apply to 1.7. Rebase required? Wrong Branch? See http://accumulo.apache.org/git.html#contributors
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12829670/ACCUMULO-4467-1.7.v1.patch
|
| JIRA Issue | ACCUMULO-4467 |
| Console output | https://builds.apache.org/job/PreCommit-ACCUMULO-Build/43/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Random Walk broken because of unmet dependency on commons-math
> --------------------------------------------------------------
>
>                 Key: ACCUMULO-4467
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4467
>             Project: Accumulo
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 1.6.6, 1.7.2
>            Reporter: Dima Spivak
>            Assignee: Sean Busbey
>             Fix For: 1.7.3, 1.8.1, 2.0.0
>
>         Attachments: ACCUMULO-4467-1.6.v1.patch, ACCUMULO-4467-1.7.v1.patch
>
>
> When trying to run the Random Walk with {{LongEach.xml}} module, I hit a failure once
we reach the {{Shard.xml}} step:
> {code}
> 16 19:52:05,146 [randomwalk.Framework] ERROR: Error during random walk
> java.lang.Exception: Error running node Shard.xml
> 	at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
> 	at org.apache.accumulo.test.randomwalk.Framework.run(Framework.java:59)
> 	at org.apache.accumulo.test.randomwalk.Framework.main(Framework.java:119)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.accumulo.start.Main$2.run(Main.java:157)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Error running node shard.BulkInsert
> 	at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
> 	at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:283)
> 	at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:278)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> 	... 1 more
> Caused by: java.lang.Exception: Failed to run map/red verify
> 	at org.apache.accumulo.test.randomwalk.shard.BulkInsert.sort(BulkInsert.java:186)
> 	at org.apache.accumulo.test.randomwalk.shard.BulkInsert.visit(BulkInsert.java:132)
> 	... 9 more
> {code}
> Digging into YARN to see why the MR job became unhappy, I see the following:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.commons.math.stat.descriptive.SummaryStatistics
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at org.apache.accumulo.core.file.rfile.RFile$Writer.<init>(RFile.java:310)
at org.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperations.java:127)
at org.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperations.java:106)
at org.apache.accumulo.core.file.DispatchingFileFactory.openWriter(DispatchingFileFactory.java:78)
at org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat$1.write(AccumuloFileOutputFormat.java:172)
at org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat$1.write(AccumuloFileOutputFormat.java:152)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:558)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:150) at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native
Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
> {code}
> It looks like [this commit|https://github.com/apache/accumulo/commit/63a8a5d7eec7b0ab6dce107d382143562e21b527]
introduced a dependency on the {{commons-math}} JAR at runtime (in the RFiles Writer class),
but tests weren't updated to ensure that the same dependency would be put onto the classpath
of MR jobs submitted by Random Walk.
> Props to [~busbey] for helping to figure out the root cause here. On a separate note,
we may want to start running this test before releases, as it appears this regression also
snuck into 1.8.0 and at least one 1.6 release (though, since I don't have any easy way to
test this against non-1.7.2 cluster, I'm limiting the affects versions to what I've confirmed
myself). Ping [~kturner], who might know the simplest way to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message