accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Kepner <kep...@ll.mit.edu>
Subject Re: Accumulo defaults
Date Sun, 18 May 2014 02:01:47 GMT
Agreed.

On Sat, May 17, 2014 at 06:01:26PM -0400, Josh Elser wrote:
> Absolutely, if you restrict a problem, you can work around it in
> other ways. Not going to argue that.
> 
> Since this is a user list though, I got very worried seeing
> something that roughly says "I'm benchmarking Accumulo with the WALs
> off". If you're providing resiliency against data lost using other
> tactics, that's fine, I just wanted to make sure that users who read
> this thread later don't think that running tests against Accumulo
> with the WALs off is "normal".
> 
> Looking forward to see the full picture of the benchmarks!
> 
> On 5/17/14, 5:27 PM, Jeremy Kepner wrote:
> >walog provides data loss protection in a specific set of circumstances.
> >Most of our deployments are under a different set of circumstances.
> >Accumulo is only one part of our systems and we have other
> >mechanisms for protecting against the loss of data.
> >We find the walog actually becomes a bottleneck in certain circumstances
> >and so turning it off increases the overall reliability of our system.
> >
> >On Sat, May 17, 2014 at 04:27:29PM -0400, Josh Elser wrote:
> >>You're likely to lose data in *any* deployment with the walogs turned off.
> >>
> >>And, to reiterate what Sean says, I wouldn't really consider any
> >>benchmark with the walogs turned off valid except for "internal"
> >>benchmarks (ones where we evaluate components only within Accumulo
> >>for the sake of improving Accumulo itself and not comparing it to
> >>other systems).
> >>
> >>On 5/17/14, 3:30 PM, Sean Busbey wrote:
> >>>You can set both of those in the accumulo-site.xml.
> >>>
> >>>However, it's going to be difficult to use benchmarks with walogs
> >>>disabled for valid comparisons to other systems. Also you are very
> >>>likely to lose data in any significantly sized deployment.
> >>>
> >>>
> >>>
> >>>On Sat, May 17, 2014 at 1:35 PM, Kepner, Jeremy - 0553 - MITLL
> >>><kepner@ll.mit.edu <mailto:kepner@ll.mit.edu>> wrote:
> >>>
> >>>    As part of our Accumulo benchmarking we have decided to set certain
> >>>    values as defaults for all our databases:
> >>>
> >>>             tserver.compaction.minor.concurrent.max=5
> >>>             table.walog.enabled=false
> >>>
> >>>    We were wondering which file(s) we would need to modify to apply
> >>>    these defaults?
> >>>
> >>>
> >>>
> >>>
> >>>--
> >>>Sean

Mime
View raw message