accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keys Botzum <kbot...@maprtech.com>
Subject Re: Accumulo on MapR Continued - LargeRowTest
Date Wed, 18 Apr 2012 17:58:26 GMT
Eric and Keith,

I will attempt the additional tests you have suggested.

Any ideas on what to do regarding those configuration properties? With hbase in hbase-site.xml,
we set those properties and they work fine. Is there some incantation I'm missing here? I
really don't want those properties to be global as they will negatively impact performance
and are only relevant to hbase and Accumulo.

Thanks,
Keys
________________________________
Keys Botzum
Senior Principal Technologist
WW Systems Engineering
kbotzum@maprtech.com
443-718-0098
MapR Technologies
http://www.mapr.com



On Apr 18, 2012, at 1:42 PM, Keith Turner wrote:

> Settings in accumulo-site.xml do not end up in the hadoop config
> object, so setting them will probably have no effect.
> 
> I would suggest running continuous ingest test and random walk test if
> you really want to stress it.  These are the test we use prior to an
> accumulo release.  You would need to exclude the random walk security
> test, it triggers known bug in 1.4 that are not fixed.
> 
> Running the test on a cluster overnight would be good.
> 
> Keith
> 
> On Wed, Apr 18, 2012 at 1:17 PM, Keys Botzum <kbotzum@maprtech.com> wrote:
>> Thanks to the help of Keith, Todd, and Eric, as well as MapR engineering, all of
the Accumulo tests is test/system/auto are now passing. Note that the latelastcontact test
only passes if I actually install zookeeper on the host. This is because of the dependency
on zkCli.sh that I mentioned earlier.
>> 
>> The final piece of the puzzle was that MapR does aggressive read ahead caching of
data as well as aggregation of writes to improve performance. As with Hbase, we don't think
this type of behavior is helpful with something like Accumulo. In our specific case, the interaction
between Accumulo and MapR's behavior results in the large row test failing.
>> 
>> So now I have one more question. To disable the caching and aggregation behavior,
we need to set these properties:
>> <property>
>> <name>fs.mapr.readbuffering</name>
>> <value>false</value>
>> </property>
>> 
>> <property>
>> <name>fs.mapr.aggregate.writes</name>
>> <value>false</value>
>> </property>
>> 
>> If I set them in core-site.xml they of course work but that's a global setting. I
want to only affect Accumulo. If I set them in accumulo-site.xml, I presume they take effect
for normal Accumulo usage, but I'm nearly certain that settings in accumulo-site.xml do not
affect the tests as I posted earlier. How can I set those two properties in a way that will
cause the tests temporary configuration to take them into account? I tried editing TestUtils.py
TestUtilsMixin settings as did work for the Accumulo property table.file.compress.type, but
the MapR related properties don't seem to take. Ideas?
>> 
>> Also, if all of the auto tests pass successfully do you feel comfortable that the
testing was sufficient or do you recommend running additional tests?
>> 
>> Thanks!
>> Keys
>> ________________________________
>> Keys Botzum
>> Senior Principal Technologist
>> WW Systems Engineering
>> kbotzum@maprtech.com
>> 443-718-0098
>> MapR Technologies
>> http://www.mapr.com
>> 


Mime
View raw message