accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keys Botzum <kbot...@maprtech.com>
Subject Re: Accumulo on MapR Continued
Date Wed, 11 Apr 2012 13:14:15 GMT
A minor correction on #3 below.  I was talking with a colleague and he pointed out that /opt/mapr/lib
does not need to be on the class path as Hadoop programs only need Hadoop classes on their
paths. So #3 below becomes:


> 3) simple.dynamic.DynamicClassloader fails with class loader issues referencing both
Accumulo and Hadoop classes. I examined simple/dynamic.py and determined that the class path
references weren't sufficient where javac was invoked. I added the following to the class
path command line options for javac and now it works:  /opt/accumulo-1.4.0/lib/*:/opt/mapr/hadoop/hadoop-0.20.2/lib/*


Keys
________________________________
Keys Botzum
Senior Principal Technologist
WW Systems Engineering
kbotzum@maprtech.com
443-718-0098
MapR Technologies
http://www.mapr.com



On Apr 10, 2012, at 1:24 PM, Keys Botzum wrote:

> First, I want to thank Billie, Todd, and Eric for their help so far. It is much appreciated.
> 
> Now, on to the results. As I stated before I'm attempting to run the Accumulo auto tests
from test/system/auto. When run standalone, most tests complete successfully on MapR but there
have been some issues. That said, I'm really glad that Accumulo has all of these tests. Very
useful for verification.
> 
> First, we found one item that required a change to MapR:
> 
> simple.bulkSplitOptimization.BulkSplitOptimizationTest failed with a seek error which
has now been addressed thanks to Billie's help. This is MapR change 6539 and we'll put that
into a release soon.
> 
> Secondly, the Accumulo tests seem to have various minor glitches. I've documented them
here for reference. I hope this is useful to the Accumulo community. I could open JIRA defects
if you want but I don't have an account on the tracking system. Just let me know.
> 
> 1) simple.examples.Examples was failing with a class loader issue. By examining simple/examples.py
I was able to determine that the package names were not correct when referring to certain
subtests, such as RandomBatchScanner. Specifically it referred to org.apache.accumulo.examples.client.RandomBatchScanner
rather than the appropriate org.apache.accumulo.examples.simple.client.RandomBatchScanner.
The same is true for RandomBatchWriter.
> 
> 2) simple.mapreduce.MapReduceTest fails because it assumes ZOOKEEPER_HOME is set as an
environment variable. Most of the other scripts appear to get this information from TestUtils.py.
The workaround is simply to set that variable before using run.py.
> 
> 3) simple.dynamic.DynamicClassloader fails with class loader issues referencing both
Accumulo and Hadoop classes. I examined simple/dynamic.py and determined that the class path
references weren't sufficient where javac was invoked. I added the following to the class
path command line options for javac and now it works:  /opt/accumulo-1.4.0/lib/*:/opt/mapr/lib/*:/opt/mapr/hadoop/hadoop-0.20.2/lib/*
> 
> Obviously that's a bit of a hack. I'm sure someone more familiar with the code can do
this better.
> 
> 4) stress.weird.LateLastContact fails with a ZOOKEEPER_HOME reference just like MapReduceTest.
> 
> 5) simple.deleterows.DeleteRowsSplitTest fails with a timeout. The test is coded to wait
120 seconds which wasn't' quite long enough on my system. I changed it to 180 seconds and
it finishes cleanly after 138 seconds. Is this typical or expected? 
> 
> 6) simple.zooCacheTest.ZooCacheTest has one minor issue. The issue is that the test times
out on my system after 60 seconds. I changed the timeout to 120 seconds and the test completes
successfully in 95 seconds. Is this typical or expected?
> 
> 7) MapR does not normally install a full zookeeper install on each node. Instead we install
a zookeeper "client" on each node which is basically the zookeeper JAR file - /opt/mapr/lib/zookeeper-3.3.2.jar.
If I set ZOOKEEPER_HOME to /opt/mapr/lib almost everything works fine since Accumulo seems
to only need the JAR file at that location. The one exception is the stress.weird.LateLastContact
which directly references zkClient.sh (in stress/weird.py) which isn't part of the MapR install.
I wanted your opinion on this. Is the test's behavior crucial? Does Accumulo need more of
zookeeper than the JAR? I want Accumulo to work,  so if there really is a strong need for
more of Zookeeper I'll recommend we make adjustments to the product once we understand the
requirement.
> 
> Third, there are still two tests that are failing when run in standalone mode and I haven't
determined the issue: simple.largeRow.LargeRowTest and simple.batchScanSplit.BatchScanSplitTest.
I will post details on those in a moment as separate messages as this email is long enough.
> 
> Thanks again for all of your help so far,
> Keys
> ________________________________
> Keys Botzum
> Senior Principal Technologist
> WW Systems Engineering
> kbotzum@maprtech.com
> 443-718-0098
> MapR Technologies
> http://www.mapr.com
> 
> 


Mime
View raw message