accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3246) More insight into native map utilization/usage
Date Tue, 21 Oct 2014 20:07:35 GMT


Eric Newton commented on ACCUMULO-3246:

Let's not attribute too much wisdom to my comments here.  Experiments should be done, and
metrics exposed.  But, the IMM / #activeTablets should be chunky.  How chunky?  I'm going
with an HDFS block size.

If HDFS block size is 100K, and there are ~10 tablets/server, let's go with 1M... maybe more
since the data is compressed on the way out.   What's the compression rate for your flush
files?  10-1?  5-1? That would be good to know.

> More insight into native map utilization/usage
> ----------------------------------------------
>                 Key: ACCUMULO-3246
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Josh Elser
>             Fix For: 1.7.0
> I often find that I choose a value for the size of the native map out of the air without
really having a good understanding of why I chose it  (aside from considerations of table.compaction.minor.logs.threshold
and tserver.walog.max.size).
> We don't have any insight into some basic metrics on the native maps. It would be nice
to be able to answer questions like
> * What is the utilization (space) of the native maps for a server
> * How much time is the server spending writing data as opposed to allocating new blocks
> I'm sure there are some other questions too.

This message was sent by Atlassian JIRA

View raw message