accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject Re: accumulo-env.sh and accumulo-site.xml recommendations for Tablet Servers
Date Mon, 27 Aug 2012 10:26:02 GMT
Increasing the size of the tserver and giving it larger caches will improve
query performance.

So, if you go up to 12G for your tserver, and increase your index/data
cache to 3/8G respectively, you should see improvements on warm queries.
 Keep an eye on your java gc timings, though.

Increasing the internal memory map size will improve write speed and
decrease compactions, but not by very much.

If you have a lot of tablets/files/servers, making the GC larger will keep
collection efficient.  Basically, if you can't keep the list of all files
which are candidates for deletion in memory, it has to use multiple passes
to reclaim files.

There is little benefit in making the other processes larger.

-Eric

On Sat, Aug 25, 2012 at 4:54 PM, Jason Trost <jason.trost@gmail.com> wrote:

> I am running Accumulo on some boxes with 32GB RAM (machine functions
> as DataNode, TaskTracker, and TabletServer).  I have them working and
> configured based on this, but I was curious if there are any rules of
> thumb for configuring the memory constrained settings for Accumulo
> Tablet servers.
>
> If I wanted to devote 16GB to the various accumulo services running on
> these boxes, what are the recommended RAM constrained settings for the
> following configs?  (Please assume accumulo is being used for Keys
> less than 100 bytes and Values less than 700 bytes).
>
> From accumulo-env.sh (taken from the 3GB example):
>
> test -z "$ACCUMULO_TSERVER_OPTS" && export
> ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx1g -Xms1g -Xss160k"
> test -z "$ACCUMULO_MASTER_OPTS"  && export
> ACCUMULO_MASTER_OPTS="${POLICY} -Xmx1g -Xms1g"
> test -z "$ACCUMULO_MONITOR_OPTS" && export
> ACCUMULO_MONITOR_OPTS="${POLICY} -Xmx1g -Xms256m"
> test -z "$ACCUMULO_GC_OPTS"      && export ACCUMULO_GC_OPTS="-Xmx256m
> -Xms256m"
> test -z "$ACCUMULO_LOGGER_OPTS"  && export
> ACCUMULO_LOGGER_OPTS="-Xmx1g -Xms256m"
> test -z "$ACCUMULO_GENERAL_OPTS" && export
> ACCUMULO_GENERAL_OPTS="-XX:+UseConcMarkSweepGC
> -XX:CMSInitiatingOccupancyFraction=75"
> test -z "$ACCUMULO_OTHER_OPTS"   && export ACCUMULO_OTHER_OPTS="-Xmx1g
> -Xms256m"
>
> And from accumulo-site.xml (taken from the 3GB example):
>
>     <property>
>       <name>tserver.memory.maps.max</name>
>       <value>1G</value>
>     </property>
>
>     <property>
>       <name>tserver.cache.data.size</name>
>       <value>50M</value>
>     </property>
>
>     <property>
>       <name>tserver.cache.index.size</name>
>       <value>100M</value>
>     </property>
>
> Are there any others that should be changed from accumulo-site.xml?
>
> Thanks,
>
> --Jason
>

Mime
View raw message