accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Accumulo defaults
Date Sat, 17 May 2014 22:01:26 GMT
Absolutely, if you restrict a problem, you can work around it in other 
ways. Not going to argue that.

Since this is a user list though, I got very worried seeing something 
that roughly says "I'm benchmarking Accumulo with the WALs off". If 
you're providing resiliency against data lost using other tactics, 
that's fine, I just wanted to make sure that users who read this thread 
later don't think that running tests against Accumulo with the WALs off 
is "normal".

Looking forward to see the full picture of the benchmarks!

On 5/17/14, 5:27 PM, Jeremy Kepner wrote:
> walog provides data loss protection in a specific set of circumstances.
> Most of our deployments are under a different set of circumstances.
> Accumulo is only one part of our systems and we have other
> mechanisms for protecting against the loss of data.
> We find the walog actually becomes a bottleneck in certain circumstances
> and so turning it off increases the overall reliability of our system.
>
> On Sat, May 17, 2014 at 04:27:29PM -0400, Josh Elser wrote:
>> You're likely to lose data in *any* deployment with the walogs turned off.
>>
>> And, to reiterate what Sean says, I wouldn't really consider any
>> benchmark with the walogs turned off valid except for "internal"
>> benchmarks (ones where we evaluate components only within Accumulo
>> for the sake of improving Accumulo itself and not comparing it to
>> other systems).
>>
>> On 5/17/14, 3:30 PM, Sean Busbey wrote:
>>> You can set both of those in the accumulo-site.xml.
>>>
>>> However, it's going to be difficult to use benchmarks with walogs
>>> disabled for valid comparisons to other systems. Also you are very
>>> likely to lose data in any significantly sized deployment.
>>>
>>>
>>>
>>> On Sat, May 17, 2014 at 1:35 PM, Kepner, Jeremy - 0553 - MITLL
>>> <kepner@ll.mit.edu <mailto:kepner@ll.mit.edu>> wrote:
>>>
>>>     As part of our Accumulo benchmarking we have decided to set certain
>>>     values as defaults for all our databases:
>>>
>>>              tserver.compaction.minor.concurrent.max=5
>>>              table.walog.enabled=false
>>>
>>>     We were wondering which file(s) we would need to modify to apply
>>>     these defaults?
>>>
>>>
>>>
>>>
>>> --
>>> Sean

Mime
View raw message