accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Park (JIRA)" <>
Subject [jira] [Created] (ACCUMULO-2889) Batch metadata table updates for new walogs
Date Tue, 10 Jun 2014 20:51:01 GMT
Jonathan Park created ACCUMULO-2889:

             Summary: Batch metadata table updates for new walogs
                 Key: ACCUMULO-2889
             Project: Accumulo
          Issue Type: Bug
    Affects Versions: 1.5.1
            Reporter: Jonathan Park

Currently, when we update the Metadata table with new loggers, we will update the metadata
for each tablet serially. We could optimize this to instead use a batchwriter to send all
metadata updates for all tablets in a batch.

A few special cases include:
- What if the !METADATA tablet was included in the batch?
- What about the root tablet?

In one of our clusters, we're experiencing particularly slow HDFS operations leading to large
oscillations in ingest performance. We haven't isolated the cause in HDFS but when we profile
the tservers, we noticed that they were waiting for metadata table operations to complete.
This would target the waiting.

Potential downsides:
Given the existing locking scheme, it looks like we may have to lock a tablet for slightly
longer (we'll lock for the duration of the batch).

This message was sent by Atlassian JIRA

View raw message