accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: Accumulo on MapR Continued - LargeRowTest
Date Wed, 18 Apr 2012 17:42:40 GMT
Settings in accumulo-site.xml do not end up in the hadoop config
object, so setting them will probably have no effect.

I would suggest running continuous ingest test and random walk test if
you really want to stress it.  These are the test we use prior to an
accumulo release.  You would need to exclude the random walk security
test, it triggers known bug in 1.4 that are not fixed.

Running the test on a cluster overnight would be good.

Keith

On Wed, Apr 18, 2012 at 1:17 PM, Keys Botzum <kbotzum@maprtech.com> wrote:
> Thanks to the help of Keith, Todd, and Eric, as well as MapR engineering, all of the
Accumulo tests is test/system/auto are now passing. Note that the latelastcontact test only
passes if I actually install zookeeper on the host. This is because of the dependency on zkCli.sh
that I mentioned earlier.
>
> The final piece of the puzzle was that MapR does aggressive read ahead caching of data
as well as aggregation of writes to improve performance. As with Hbase, we don't think this
type of behavior is helpful with something like Accumulo. In our specific case, the interaction
between Accumulo and MapR's behavior results in the large row test failing.
>
> So now I have one more question. To disable the caching and aggregation behavior, we
need to set these properties:
> <property>
> <name>fs.mapr.readbuffering</name>
> <value>false</value>
> </property>
>
> <property>
> <name>fs.mapr.aggregate.writes</name>
> <value>false</value>
> </property>
>
> If I set them in core-site.xml they of course work but that's a global setting. I want
to only affect Accumulo. If I set them in accumulo-site.xml, I presume they take effect for
normal Accumulo usage, but I'm nearly certain that settings in accumulo-site.xml do not affect
the tests as I posted earlier. How can I set those two properties in a way that will cause
the tests temporary configuration to take them into account? I tried editing TestUtils.py
TestUtilsMixin settings as did work for the Accumulo property table.file.compress.type, but
the MapR related properties don't seem to take. Ideas?
>
> Also, if all of the auto tests pass successfully do you feel comfortable that the testing
was sufficient or do you recommend running additional tests?
>
> Thanks!
> Keys
> ________________________________
> Keys Botzum
> Senior Principal Technologist
> WW Systems Engineering
> kbotzum@maprtech.com
> 443-718-0098
> MapR Technologies
> http://www.mapr.com
>

Mime
View raw message