accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject Re: Accumulo on MapR Continued
Date Wed, 11 Apr 2012 17:26:44 GMT
Thanks Keys,

That was a lot of digging around in the tests!

Tickets made (ACCUMULO-521 - ACCUMULO-527} and fixed in the 1.4 branch and
trunk.  Let me know if you have issues.

-Eric

On Tue, Apr 10, 2012 at 4:24 PM, Eric Newton <eric.newton@gmail.com> wrote:

> This is great!  I'll make tickets.
>
> -Eric
>
>
> On Tue, Apr 10, 2012 at 1:24 PM, Keys Botzum <kbotzum@maprtech.com> wrote:
>
>> First, I want to thank Billie, Todd, and Eric for their help so far. It
>> is much appreciated.
>>
>> Now, on to the results. As I stated before I'm attempting to run the
>> Accumulo auto tests from test/system/auto. When run standalone, most tests
>> complete successfully on MapR but there have been some issues. That said,
>> I'm really glad that Accumulo has all of these tests. Very useful for
>> verification.
>>
>> First, we found one item that required a change to MapR:
>>
>> simple.bulkSplitOptimization.BulkSplitOptimizationTest failed with a seek
>> error which has now been addressed thanks to Billie's help. This is MapR
>> change 6539 and we'll put that into a release soon.
>>
>> Secondly, the Accumulo tests seem to have various minor glitches. I've
>> documented them here for reference. I hope this is useful to the Accumulo
>> community. I could open JIRA defects if you want but I don't have an
>> account on the tracking system. Just let me know.
>>
>> 1) simple.examples.Examples was failing with a class loader issue. By
>> examining simple/examples.py I was able to determine that the package names
>> were not correct when referring to certain subtests, such as
>> RandomBatchScanner. Specifically it referred to org.apache.accumulo.examples.client.RandomBatchScanner
>> rather than the appropriate org.apache.accumulo.examples.*simple*.client.RandomBatchScanner.
>> The same is true for RandomBatchWriter.
>>
>> 2) simple.mapreduce.MapReduceTest fails because it assumes
>> ZOOKEEPER_HOME is set as an environment variable. Most of the other scripts
>> appear to get this information from TestUtils.py. The workaround is simply
>> to set that variable before using run.py.
>>
>> 3) simple.dynamic.DynamicClassloader fails with class loader issues
>> referencing both Accumulo and Hadoop classes. I examined simple/dynamic.py
>> and determined that the class path references weren't sufficient where
>> javac was invoked. I added the following to the class path command line
>> options for javac and now it works:
>>  /opt/accumulo-1.4.0/lib/*:/opt/mapr/lib/*:/opt/mapr/hadoop/hadoop-0.20.2/lib/*
>>
>> Obviously that's a bit of a hack. I'm sure someone more familiar with the
>> code can do this better.
>>
>> 4) stress.weird.LateLastContact fails with a ZOOKEEPER_HOME reference
>> just like MapReduceTest.
>>
>> 5) simple.deleterows.DeleteRowsSplitTest fails with a timeout. The test
>> is coded to wait 120 seconds which wasn't' quite long enough on my system.
>> I changed it to 180 seconds and it finishes cleanly after 138 seconds. Is
>> this typical or expected?
>>
>> 6) simple.zooCacheTest.ZooCacheTest has one minor issue. The issue is
>> that the test times out on my system after 60 seconds. I changed the
>> timeout to 120 seconds and the test completes successfully in 95 seconds.
>> Is this typical or expected?
>>
>> 7) MapR does not normally install a full zookeeper install on each node.
>> Instead we install a zookeeper "client" on each node which is basically the
>> zookeeper JAR file - /opt/mapr/lib/zookeeper-3.3.2.jar. If I set
>> ZOOKEEPER_HOME to /opt/mapr/lib almost everything works fine since Accumulo
>> seems to only need the JAR file at that location. The one exception is
>> the stress.weird.LateLastContact which directly references zkClient.sh (in
>> stress/weird.py) which isn't part of the MapR install. I wanted your
>> opinion on this. Is the test's behavior crucial? Does Accumulo need more of
>> zookeeper than the JAR? I want Accumulo to work,  so if there really is a
>> strong need for more of Zookeeper I'll recommend we make adjustments to the
>> product once we understand the requirement.
>>
>> Third, there are still two tests that are failing when run in standalone
>> mode and I haven't determined the issue: simple.largeRow.LargeRowTest
>> and simple.batchScanSplit.BatchScanSplitTest. I will post details on
>> those in a moment as separate messages as this email is long enough.
>>
>> Thanks again for all of your help so far,
>> Keys
>> ________________________________
>> Keys Botzum
>> Senior Principal Technologist
>> WW Systems Engineering
>> kbotzum@maprtech.com
>> 443-718-0098
>> MapR Technologies
>> http://www.mapr.com
>>
>>
>>
>

Mime
View raw message