accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject Re: large values
Date Wed, 14 Mar 2012 17:22:12 GMT
It depends on how large loggers' JVM can get.  We typically run them with
less memory than the tablet servers, so we see the loggers break when the
large mutation hits the logger.

If your logger has a large JVM, then your tablet server will crash
receiving the message, or serializing it to the loggers, and then when it
tries to recover after crashing, which will slowly take down your whole
cluster.

However, if a large value is bulk loaded, you will see the tablet servers
crash when they do a major compaction.

The upper limit is something like 1/3 of the JVM memory size, but in
practice, if you have a lot of large values, this limit will be much lower.

-Eric

On Wed, Mar 14, 2012 at 1:00 PM, Aaron Cordova <aaron@cordovas.org> wrote:

> What are the current limitations on storing large values in Accumulo?
> Specifically, what would break first and at how big would a value have to
> be to cause the break? Also, is storing lots of large values much worse
> than storing occasional large values?
>
> I can of course test this but it'd be nice to hear what is already known
> about the subject first.
>
> Aaron

Mime
View raw message