Return-Path: 
 <notifications-return-25188-apmail-accumulo-notifications-archive=accumulo.apache.org@accumulo.apache.org>
X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org
Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 1D03F179A0
	for <apmail-accumulo-notifications-archive@minotaur.apache.org>;
 Thu, 30 Oct 2014 22:27:34 +0000 (UTC)
Received: (qmail 45719 invoked by uid 500); 30 Oct 2014 22:27:33 -0000
Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org
Received: (qmail 45681 invoked by uid 500); 30 Oct 2014 22:27:33 -0000
Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:notifications-help@accumulo.apache.org>
List-Unsubscribe: <mailto:notifications-unsubscribe@accumulo.apache.org>
List-Post: <mailto:notifications@accumulo.apache.org>
List-Id: <notifications.accumulo.apache.org>
Reply-To: jira@apache.org
Delivered-To: mailing list notifications@accumulo.apache.org
Received: (qmail 45669 invoked by uid 99); 30 Oct 2014 22:27:33 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Oct 2014 22:27:33 +0000
Date: Thu, 30 Oct 2014 22:27:33 +0000 (UTC)
From: "Josh Elser (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12751811.1414708043000.377373.1414708053612@Atlassian.JIRA>
In-Reply-To: <JIRA.12751811.1414708043000@Atlassian.JIRA>
References: <JIRA.12751811.1414708043000@Atlassian.JIRA>
 <JIRA.12751811.1414708043130@arcas>
Subject: [jira] [Created] (ACCUMULO-3282) Reduce scans of metadata when
 assigning tablet
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394

Josh Elser created ACCUMULO-3282:
------------------------------------

             Summary: Reduce scans of metadata when assigning tablet
                 Key: ACCUMULO-3282
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3282
             Project: Accumulo
          Issue Type: Improvement
          Components: tserver
            Reporter: Josh Elser
             Fix For: 1.7.0


Noticed the following case digging through the assignment code:

{code:title=TabletServer.java}
  public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent extent, TServerInstance instance, SortedMap<Key,Value> tabletsKeyValues,
      String clientAddress, ZooLock lock) throws AccumuloSecurityException, DistributedStoreException, AccumuloException {
...
  public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent extent, TServerInstance instance, SortedMap<Key,Value> tabletsKeyValues,
      String clientAddress, ZooLock lock) throws AccumuloSecurityException, DistributedStoreException, AccumuloException {

    log.debug("verifying extent " + extent);
    if (extent.isRootTablet()) {
      return verifyRootTablet(extent, instance);
    }
    String tableToVerify = MetadataTable.ID;
    if (extent.isMeta())
      tableToVerify = RootTable.ID;

    List<ColumnFQ> columnsToFetch = Arrays.asList(new ColumnFQ[] {TabletsSection.ServerColumnFamily.DIRECTORY_COLUMN,
        TabletsSection.TabletColumnFamily.PREV_ROW_COLUMN, TabletsSection.TabletColumnFamily.SPLIT_RATIO_COLUMN,
        TabletsSection.TabletColumnFamily.OLD_PREV_ROW_COLUMN, TabletsSection.ServerColumnFamily.TIME_COLUMN});

    ScannerImpl scanner = new ScannerImpl(HdfsZooInstance.getInstance(), SystemCredentials.get(), tableToVerify, Authorizations.EMPTY);
    scanner.setRange(extent.toMetadataRange());

    TreeMap<Key,Value> tkv = new TreeMap<Key,Value>();
    for (Entry<Key,Value> entry : scanner)
      tkv.put(entry.getKey(), entry.getValue());

    // only populate map after success
    if (tabletsKeyValues == null) {
      tabletsKeyValues = tkv;
    } else {
      tabletsKeyValues.clear();
      tabletsKeyValues.putAll(tkv);
{code}

Essentially, we read a few columns for the tablet's row from metadata and (when metadata is in a consistent state) we return those columns to the Tablet constructor.

{code:title=Tablet.java}
  private Tablet(TabletServer tabletServer, Text location, KeyExtent extent, TabletResourceManager trm, Configuration conf, VolumeManager fs,
      SortedMap<Key,Value> tabletsKeyValues) throws IOException {
    this(tabletServer, location, extent, trm, conf, fs, lookupLogEntries(extent, tabletsKeyValues), lookupDatafiles(tabletServer.getSystemConfiguration(), fs,
        extent, tabletsKeyValues), lookupTime(tabletServer.getSystemConfiguration(), extent, tabletsKeyValues), lookupLastServer(extent, tabletsKeyValues),
        lookupScanFiles(extent, tabletsKeyValues, fs), lookupFlushID(extent, tabletsKeyValues), lookupCompactID(extent, tabletsKeyValues));
  }
{code}

{{lookupDataFiles}} reaches back out to the metadata table to fetch this column. I'm not sure if there's a reason why we can't do this all at once.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)