kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Heo <jason.heo....@gmail.com>
Subject Re: What does RowSet Compaction Duration means?
Date Wed, 15 Mar 2017 01:10:58 GMT
Hi Alexey.

Thank you for your reply.

With your help, now I can understand what 'compact_rs_duration` means. But
the `default_num_replicas` is just 3 not 5 :(

It seems compaction on tableB affects huge on bulk loading on tableA. Is
there a way to minimize compaction activities? (something like changing
configuration of Kudu)

The FAQ says that "Since compactions are so predictable, the only tuning
knob available is the number of threads dedicated to flushes and
compactions in the *maintenance manager*."

my `maintenance_manager_num_threads` is already 1.

Thanks.

2017-03-15 3:48 GMT+09:00 Alexey Serbin <aserbin@cloudera.com>:

> Hi Jason,
>
> As I understand, that 'milliseconds / second' cryptic unit means 'number
> of units / for sampling (or averaging) interval'.
>
> I.e., they capture that metric reading (expressed in milliseconds) every
> second, subtract previous value from the current value, and declare the
> result as the result measurement at current time.  If not capturing every
> second, then it's about measuring every X seconds, do the subtraction of
> the previous from the current measurement, and then divide by X.
>
> For a single tablet, the 'compact_rs_duration' metric stands for 'Time
> spent compacting RowSets'.  As I understand, that 'total_kudu_compact_rs_
> duration_sum_rate_across_kudu_replicas' is sum/accumulation of those
> measurements for all existing replicas of the specified tablet across Kudu
> cluster.
>
> I suspect you have the replication factor of 5 for that tablet, and at
> some point all replicas become busy with rowset compaction all the time.
>
> Compactions on tables are run in the background.  Compactions on different
> tables run independently.  So, if you have some other activity doing
> inserts/updates on tableB, then it's natural to see compaction happen on
> tabletB as well.
>
>
> Best regards,
>
> Alexey
>
> On Tue, Mar 14, 2017 at 12:50 AM, Jason Heo <jason.heo.sde@gmail.com>
> wrote:
>
>> Hi.
>>
>> I'm stuck with performance degradation on compaction happens.
>>
>> My Duration is "4956.71 milliseconds / second" What does this mean? I
>> can't figure it out.
>>
>> Here is the captured image: http://imgur.com/WU9sRRq
>>
>> When I'm doing bulk indexing on tableA, sometimes compaction happens over
>> tableB. Is this situation is natural?
>>
>> Thanks.
>>
>
>

Mime
View raw message