accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-4112) MinC start/stop updates are always hsync'd
Date Thu, 14 Jan 2016 22:23:39 GMT


Eric Newton commented on ACCUMULO-4112:

I wrote a little MAC test for this scenario:

* Lots of tablets per server.
* Write lots of mutations uniformly over the tablets.
* Lots of minor compaction threads.
* Small in-memory map.

If I change the Durability to LOG, there's no improvement in performance.

I had to tweak the test a bit in order to not overwhelm MiniDFS.

I'll attach the little test. Perhaps there's something that fluo is doing differently?

BTW, if you are running with something less than 1.8.0-SNAPSHOT you are probably seeing the
effect of updating the metadata table with WAL markers (ACCUMULO-3423). The only solution
is to use fewer tablets (or release 1.8.0).

> MinC start/stop updates are always hsync'd
> ------------------------------------------
>                 Key: ACCUMULO-4112
>                 URL:
>             Project: Accumulo
>          Issue Type: Bug
>          Components: tserver
>         Environment: Fluo testing on a 20-node cluster
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>            Priority: Minor
> [~kturner] writes:
> {quote}
> I was running a Fluo test with 1.8.0-SNAP on my workstation.  My Fluo table had a ton
of tablets.   I was seeing terrible performance.   I started looking at the tserver and noticed
it was always calling hsync.  I tracked down the problem to the fact that when minc start
and stop events are written to the log they are always written w/ sync level.   My poor little
tserver was constantly minor compacting (probably had around 600 tablets that were all being
written to).  
> I changed the test config to create like 15 tablets and the performance was much better.
 All cores were 100% utilized, which was not the case when hsync was always called.
> {quote}

This message was sent by Atlassian JIRA

View raw message