accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-1905) WALog is slow
Date Mon, 18 Nov 2013 23:31:20 GMT


Eric Newton commented on ACCUMULO-1905:

Analysis shows that every group sync (using hsync in hadoop 2.2.0), was taking 50-100ms. 
If there wasn't enough data buffered in the group, then smallish amounts were being sent.

First, I increased the client batch size, which is hard-coded (to 1<<17 bytes).  This
increased the performance to the same level as the plain sync call available in hadoop 1.2.
 Also, adding a second client increased the performance since there were now more clients
participating in the group sync.

Finally, I put the client buffer setting back down, and increased the size of the session
buffer through configuration:

root@test> config -s tserver.mutation.queue.max=2M

Now, without any code changes, performance is back up to 40K k-v/sec with a single client.

I was using a very small in-memory map (10M), which caused lots of compactions, which, in
turn, caused lots of flushed writes to the !METADATA table.  Those writes would push through
to the WAL, which always took 50-100ms.

So, after increasing the IMM to 7G, performance was steady at 73K k-v/sec.

> WALog is slow
> -------------
>                 Key: ACCUMULO-1905
>                 URL:
>             Project: Accumulo
>          Issue Type: Bug
>          Components: tserver
>         Environment: hadoop 2.2.0, accumulo 1.6.0, hash 45fbee6937549048c74fe176f201c246de0f5e0a
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>            Priority: Critical
>             Fix For: 1.6.0
> I thought that the continuous ingest test was going slower than it used to.  I was getting
about 20K k-v/sec on my desktop.
> After turning off the WALog, ingest shot up to 200K k-v/sec.  I expected some improvement,
but not an order of magnitude.

This message was sent by Atlassian JIRA

View raw message