Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1D7BC11210 for ; Tue, 10 Jun 2014 20:51:03 +0000 (UTC) Received: (qmail 47451 invoked by uid 500); 10 Jun 2014 20:51:02 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 47401 invoked by uid 500); 10 Jun 2014 20:51:02 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 47392 invoked by uid 99); 10 Jun 2014 20:51:02 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jun 2014 20:51:02 +0000 Date: Tue, 10 Jun 2014 20:51:01 +0000 (UTC) From: "Jonathan Park (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (ACCUMULO-2889) Batch metadata table updates for new walogs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Jonathan Park created ACCUMULO-2889: --------------------------------------- Summary: Batch metadata table updates for new walogs Key: ACCUMULO-2889 URL: https://issues.apache.org/jira/browse/ACCUMULO-2889 Project: Accumulo Issue Type: Bug Affects Versions: 1.5.1 Reporter: Jonathan Park Currently, when we update the Metadata table with new loggers, we will update the metadata for each tablet serially. We could optimize this to instead use a batchwriter to send all metadata updates for all tablets in a batch. A few special cases include: - What if the !METADATA tablet was included in the batch? - What about the root tablet? Benefit: In one of our clusters, we're experiencing particularly slow HDFS operations leading to large oscillations in ingest performance. We haven't isolated the cause in HDFS but when we profile the tservers, we noticed that they were waiting for metadata table operations to complete. This would target the waiting. Potential downsides: Given the existing locking scheme, it looks like we may have to lock a tablet for slightly longer (we'll lock for the duration of the batch). -- This message was sent by Atlassian JIRA (v6.2#6252)