Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Thu, 6 Nov 2014 17:50:34 +0000 (UTC)
From: "Josh Elser (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12753219.1415229633000.435745.1415296234432@Atlassian.JIRA>
In-Reply-To: <JIRA.12753219.1415229633000@Atlassian.JIRA>
References: <JIRA.12753219.1415229633000@Atlassian.JIRA>
 <JIRA.12753219.1415229633994@arcas>
Subject: [jira] [Comment Edited] (ACCUMULO-3303) funky performance with
 large WAL
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200530#comment-14200530 ] 

Josh Elser edited comment on ACCUMULO-3303 at 11/6/14 5:49 PM:
---------------------------------------------------------------

bq. Logically, it makes sense, but it might not be obvious to people that haven't thought about all of the implications of WAL sizing.

We (at least) have the descriptions on in the [user manual|http://accumulo.apache.org/1.6/accumulo_user_manual.html] on tserver.memory.maps.max and tserver.walog.max.size. I'll open an issue to add a section to the user manual that can specifically cover this subject (since the two sentences don't quite give it enough justice). ACCUMULO-3308


was (Author: elserj):
bq. Logically, it makes sense, but it might not be obvious to people that haven't thought about all of the implications of WAL sizing.

We (at least) have the descriptions on in the [user manual|http://accumulo.apache.org/1.6/accumulo_user_manual.html] on tserver.memory.maps.max and tserver.walog.max.size. I'll open an issue to add a section to the user manual that can specifically cover this subject (since the two sentences don't quite give it enough justice).

> funky performance with large WAL
> --------------------------------
>
>                 Key: ACCUMULO-3303
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3303
>             Project: Accumulo
>          Issue Type: Bug
>          Components: logger, tserver
>    Affects Versions: 1.6.1
>            Reporter: Adam Fuchs
>         Attachments: 1GB_WAL.png, 2GB_WAL.png, 4GB_WAL.png, 512MB_WAL.png, 8GB_WAL.png, WAL_disabled.png
>
>
> The tserver seems to get into a funky state when writing to a large write-ahead log. I ran some continuous ingest tests varying tserver.walog.max.size in {512M, 1G, 2G, 4G, 8G} and got some results that I have yet to understand. I was expecting to see the effects of walog metadata management as described in ACCUMULO-2889, but I also found an additional behavior of ingest slowing down for long periods when using a large walog size.
> The cluster configuration was as follows:
> {code}
> Accumulo version: 1.6.2-SNAPSHOT (current head of origin/1.6)
> Nodes: 4
> Masters: 1
> Slaves: 3
> Cores per node: 24
> Drives per node: 8x1TB data + 2 raided system
> Memory per node: 64GB
> tserver.memory.maps.max=2G
> table.file.compress.type=snappy (for ci table only)
> tserver.mutation.queue.max=16M
> tserver.wal.sync.method=hflush
> Native maps enabled
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)