Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Wed, 21 Sep 2016 20:49:20 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.13006044.1474321326000.631019.1474490960935@Atlassian.JIRA>
In-Reply-To: <JIRA.13006044.1474321326000@Atlassian.JIRA>
References: <JIRA.13006044.1474321326000@Atlassian.JIRA> <JIRA.13006044.1474321326807@arcas>
Subject: [jira] [Commented] (ACCUMULO-4467) Random Walk broken because of
 unmet dependency on commons-math
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 21 Sep 2016 20:49:23 -0000


    [ https://issues.apache.org/jira/browse/ACCUMULO-4467?page=3Dcom.atlass=
ian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1=
5511131#comment-15511131 ]=20

Hadoop QA commented on ACCUMULO-4467:
-------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {col=
or} | {color:red} ACCUMULO-4467 does not apply to 1.8. Rebase required? Wro=
ng Branch? See http://accumulo.apache.org/git.html#contributors for help. {=
color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/1282967=
9/ACCUMULO-4467-1.8.v1.patch |
| JIRA Issue | ACCUMULO-4467 |
| Console output | https://builds.apache.org/job/PreCommit-ACCUMULO-Build/4=
4/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.


> Random Walk broken because of unmet dependency on commons-math
> --------------------------------------------------------------
>
>                 Key: ACCUMULO-4467
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4467
>             Project: Accumulo
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 1.6.6, 1.7.2
>            Reporter: Dima Spivak
>            Assignee: Sean Busbey
>             Fix For: 1.7.3, 1.8.1, 2.0.0
>
>         Attachments: ACCUMULO-4467-1.6.v1.patch, ACCUMULO-4467-1.7.v1.pat=
ch, ACCUMULO-4467-1.8.v1.patch
>
>
> When trying to run the Random Walk with {{LongEach.xml}} module, I hit a =
failure once we reach the {{Shard.xml}} step:
> {code}
> 16 19:52:05,146 [randomwalk.Framework] ERROR: Error during random walk
> java.lang.Exception: Error running node Shard.xml
> =09at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
> =09at org.apache.accumulo.test.randomwalk.Framework.run(Framework.java:59=
)
> =09at org.apache.accumulo.test.randomwalk.Framework.main(Framework.java:1=
19)
> =09at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> =09at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImp=
l.java:57)
> =09at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcc=
essorImpl.java:43)
> =09at java.lang.reflect.Method.invoke(Method.java:606)
> =09at org.apache.accumulo.start.Main$2.run(Main.java:157)
> =09at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Error running node shard.BulkInsert
> =09at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
> =09at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:283)
> =09at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:278)
> =09at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> =09at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:=
471)
> =09at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto=
r.java:1145)
> =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut=
or.java:615)
> =09at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.j=
ava:35)
> =09... 1 more
> Caused by: java.lang.Exception: Failed to run map/red verify
> =09at org.apache.accumulo.test.randomwalk.shard.BulkInsert.sort(BulkInser=
t.java:186)
> =09at org.apache.accumulo.test.randomwalk.shard.BulkInsert.visit(BulkInse=
rt.java:132)
> =09... 9 more
> {code}
> Digging into YARN to see why the MR job became unhappy, I see the followi=
ng:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.commons.math.stat.des=
criptive.SummaryStatistics at java.net.URLClassLoader$1.run(URLClassLoader.=
java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java=
.security.AccessController.doPrivileged(Native Method) at java.net.URLClass=
Loader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClas=
s(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launc=
her.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at o=
rg.apache.accumulo.core.file.rfile.RFile$Writer.<init>(RFile.java:310) at o=
rg.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperatio=
ns.java:127) at org.apache.accumulo.core.file.rfile.RFileOperations.openWri=
ter(RFileOperations.java:106) at org.apache.accumulo.core.file.DispatchingF=
ileFactory.openWriter(DispatchingFileFactory.java:78) at org.apache.accumul=
o.core.client.mapreduce.AccumuloFileOutputFormat$1.write(AccumuloFileOutput=
Format.java:172) at org.apache.accumulo.core.client.mapreduce.AccumuloFileO=
utputFormat$1.write(AccumuloFileOutputFormat.java:152) at org.apache.hadoop=
.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:558) at or=
g.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOu=
tputContextImpl.java:89) at org.apache.hadoop.mapreduce.lib.reduce.WrappedR=
educer$Context.write(WrappedReducer.java:105) at org.apache.hadoop.mapreduc=
e.Reducer.reduce(Reducer.java:150) at org.apache.hadoop.mapreduce.Reducer.r=
un(Reducer.java:171) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(R=
educeTask.java:627) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.j=
ava:389) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at=
 java.security.AccessController.doPrivileged(Native Method) at javax.securi=
ty.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGr=
oupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.ma=
pred.YarnChild.main(YarnChild.java:158)=20
> {code}
> It looks like [this commit|https://github.com/apache/accumulo/commit/63a8=
a5d7eec7b0ab6dce107d382143562e21b527] introduced a dependency on the {{comm=
ons-math}} JAR at runtime (in the RFiles Writer class), but tests weren't u=
pdated to ensure that the same dependency would be put onto the classpath o=
f MR jobs submitted by Random Walk.
> Props to [~busbey] for helping to figure out the root cause here. On a se=
parate note, we may want to start running this test before releases, as it =
appears this regression also snuck into 1.8.0 and at least one 1.6 release =
(though, since I don't have any easy way to test this against non-1.7.2 clu=
ster, I'm limiting the affects versions to what I've confirmed myself). Pin=
g [~kturner], who might know the simplest way to fix this.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)