Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Date: Sat, 17 May 2014 22:01:47 -0400
From: Jeremy Kepner <kepner@ll.mit.edu>
To: <user@accumulo.apache.org>
Subject: Re: Accumulo defaults
Message-ID: <20140518020147.GA10757@ll.mit.edu>
Reply-To: <kepner@ll.mit.edu>
References: <2AA25187-A597-4869-A5A1-548137BF0A13@ll.mit.edu>
 <129A40C3-6F6C-4552-9B20-9A7BB2EB7D12@ll.mit.edu>
 <CAGHyZ6JzwR8HMJW91Z=A7wWTNxoosyVRjvNeus+cMio5XagUrA@mail.gmail.com>
 <5377C631.8070501@gmail.com>
 <20140517212700.GA9012@ll.mit.edu>
 <5377DC36.4070807@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <5377DC36.4070807@gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)

Agreed.

On Sat, May 17, 2014 at 06:01:26PM -0400, Josh Elser wrote:
> Absolutely, if you restrict a problem, you can work around it in
> other ways. Not going to argue that.
> 
> Since this is a user list though, I got very worried seeing
> something that roughly says "I'm benchmarking Accumulo with the WALs
> off". If you're providing resiliency against data lost using other
> tactics, that's fine, I just wanted to make sure that users who read
> this thread later don't think that running tests against Accumulo
> with the WALs off is "normal".
> 
> Looking forward to see the full picture of the benchmarks!
> 
> On 5/17/14, 5:27 PM, Jeremy Kepner wrote:
> >walog provides data loss protection in a specific set of circumstances.
> >Most of our deployments are under a different set of circumstances.
> >Accumulo is only one part of our systems and we have other
> >mechanisms for protecting against the loss of data.
> >We find the walog actually becomes a bottleneck in certain circumstances
> >and so turning it off increases the overall reliability of our system.
> >
> >On Sat, May 17, 2014 at 04:27:29PM -0400, Josh Elser wrote:
> >>You're likely to lose data in *any* deployment with the walogs turned off.
> >>
> >>And, to reiterate what Sean says, I wouldn't really consider any
> >>benchmark with the walogs turned off valid except for "internal"
> >>benchmarks (ones where we evaluate components only within Accumulo
> >>for the sake of improving Accumulo itself and not comparing it to
> >>other systems).
> >>
> >>On 5/17/14, 3:30 PM, Sean Busbey wrote:
> >>>You can set both of those in the accumulo-site.xml.
> >>>
> >>>However, it's going to be difficult to use benchmarks with walogs
> >>>disabled for valid comparisons to other systems. Also you are very
> >>>likely to lose data in any significantly sized deployment.
> >>>
> >>>
> >>>
> >>>On Sat, May 17, 2014 at 1:35 PM, Kepner, Jeremy - 0553 - MITLL
> >>><kepner@ll.mit.edu <mailto:kepner@ll.mit.edu>> wrote:
> >>>
> >>>    As part of our Accumulo benchmarking we have decided to set certain
> >>>    values as defaults for all our databases:
> >>>
> >>>             tserver.compaction.minor.concurrent.max=5
> >>>             table.walog.enabled=false
> >>>
> >>>    We were wondering which file(s) we would need to modify to apply
> >>>    these defaults?
> >>>
> >>>
> >>>
> >>>
> >>>--
> >>>Sean