accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keys Botzum <kbot...@maprtech.com>
Subject Re: Accumulo on MapR Continued
Date Tue, 10 Apr 2012 17:24:37 GMT
First, I want to thank Billie, Todd, and Eric for their help so far. It is much appreciated.

Now, on to the results. As I stated before I'm attempting to run the Accumulo auto tests from
test/system/auto. When run standalone, most tests complete successfully on MapR but there
have been some issues. That said, I'm really glad that Accumulo has all of these tests. Very
useful for verification.

First, we found one item that required a change to MapR:

simple.bulkSplitOptimization.BulkSplitOptimizationTest failed with a seek error which has
now been addressed thanks to Billie's help. This is MapR change 6539 and we'll put that into
a release soon.

Secondly, the Accumulo tests seem to have various minor glitches. I've documented them here
for reference. I hope this is useful to the Accumulo community. I could open JIRA defects
if you want but I don't have an account on the tracking system. Just let me know.

1) simple.examples.Examples was failing with a class loader issue. By examining simple/examples.py
I was able to determine that the package names were not correct when referring to certain
subtests, such as RandomBatchScanner. Specifically it referred to org.apache.accumulo.examples.client.RandomBatchScanner
rather than the appropriate org.apache.accumulo.examples.simple.client.RandomBatchScanner.
The same is true for RandomBatchWriter.

2) simple.mapreduce.MapReduceTest fails because it assumes ZOOKEEPER_HOME is set as an environment
variable. Most of the other scripts appear to get this information from TestUtils.py. The
workaround is simply to set that variable before using run.py.

3) simple.dynamic.DynamicClassloader fails with class loader issues referencing both Accumulo
and Hadoop classes. I examined simple/dynamic.py and determined that the class path references
weren't sufficient where javac was invoked. I added the following to the class path command
line options for javac and now it works:  /opt/accumulo-1.4.0/lib/*:/opt/mapr/lib/*:/opt/mapr/hadoop/hadoop-0.20.2/lib/*

Obviously that's a bit of a hack. I'm sure someone more familiar with the code can do this
better.

4) stress.weird.LateLastContact fails with a ZOOKEEPER_HOME reference just like MapReduceTest.

5) simple.deleterows.DeleteRowsSplitTest fails with a timeout. The test is coded to wait 120
seconds which wasn't' quite long enough on my system. I changed it to 180 seconds and it finishes
cleanly after 138 seconds. Is this typical or expected? 

6) simple.zooCacheTest.ZooCacheTest has one minor issue. The issue is that the test times
out on my system after 60 seconds. I changed the timeout to 120 seconds and the test completes
successfully in 95 seconds. Is this typical or expected?

7) MapR does not normally install a full zookeeper install on each node. Instead we install
a zookeeper "client" on each node which is basically the zookeeper JAR file - /opt/mapr/lib/zookeeper-3.3.2.jar.
If I set ZOOKEEPER_HOME to /opt/mapr/lib almost everything works fine since Accumulo seems
to only need the JAR file at that location. The one exception is the stress.weird.LateLastContact
which directly references zkClient.sh (in stress/weird.py) which isn't part of the MapR install.
I wanted your opinion on this. Is the test's behavior crucial? Does Accumulo need more of
zookeeper than the JAR? I want Accumulo to work,  so if there really is a strong need for
more of Zookeeper I'll recommend we make adjustments to the product once we understand the
requirement.

Third, there are still two tests that are failing when run in standalone mode and I haven't
determined the issue: simple.largeRow.LargeRowTest and simple.batchScanSplit.BatchScanSplitTest.
I will post details on those in a moment as separate messages as this email is long enough.

Thanks again for all of your help so far,
Keys
________________________________
Keys Botzum
Senior Principal Technologist
WW Systems Engineering
kbotzum@maprtech.com
443-718-0098
MapR Technologies
http://www.mapr.com



Mime
View raw message