accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ACCUMULO-3282) Reduce scans of metadata when assigning tablet
Date Thu, 30 Oct 2014 22:27:33 GMT
Josh Elser created ACCUMULO-3282:
------------------------------------

             Summary: Reduce scans of metadata when assigning tablet
                 Key: ACCUMULO-3282
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3282
             Project: Accumulo
          Issue Type: Improvement
          Components: tserver
            Reporter: Josh Elser
             Fix For: 1.7.0


Noticed the following case digging through the assignment code:

{code:title=TabletServer.java}
  public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent extent, TServerInstance
instance, SortedMap<Key,Value> tabletsKeyValues,
      String clientAddress, ZooLock lock) throws AccumuloSecurityException, DistributedStoreException,
AccumuloException {
...
  public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent extent, TServerInstance
instance, SortedMap<Key,Value> tabletsKeyValues,
      String clientAddress, ZooLock lock) throws AccumuloSecurityException, DistributedStoreException,
AccumuloException {

    log.debug("verifying extent " + extent);
    if (extent.isRootTablet()) {
      return verifyRootTablet(extent, instance);
    }
    String tableToVerify = MetadataTable.ID;
    if (extent.isMeta())
      tableToVerify = RootTable.ID;

    List<ColumnFQ> columnsToFetch = Arrays.asList(new ColumnFQ[] {TabletsSection.ServerColumnFamily.DIRECTORY_COLUMN,
        TabletsSection.TabletColumnFamily.PREV_ROW_COLUMN, TabletsSection.TabletColumnFamily.SPLIT_RATIO_COLUMN,
        TabletsSection.TabletColumnFamily.OLD_PREV_ROW_COLUMN, TabletsSection.ServerColumnFamily.TIME_COLUMN});

    ScannerImpl scanner = new ScannerImpl(HdfsZooInstance.getInstance(), SystemCredentials.get(),
tableToVerify, Authorizations.EMPTY);
    scanner.setRange(extent.toMetadataRange());

    TreeMap<Key,Value> tkv = new TreeMap<Key,Value>();
    for (Entry<Key,Value> entry : scanner)
      tkv.put(entry.getKey(), entry.getValue());

    // only populate map after success
    if (tabletsKeyValues == null) {
      tabletsKeyValues = tkv;
    } else {
      tabletsKeyValues.clear();
      tabletsKeyValues.putAll(tkv);
{code}

Essentially, we read a few columns for the tablet's row from metadata and (when metadata is
in a consistent state) we return those columns to the Tablet constructor.

{code:title=Tablet.java}
  private Tablet(TabletServer tabletServer, Text location, KeyExtent extent, TabletResourceManager
trm, Configuration conf, VolumeManager fs,
      SortedMap<Key,Value> tabletsKeyValues) throws IOException {
    this(tabletServer, location, extent, trm, conf, fs, lookupLogEntries(extent, tabletsKeyValues),
lookupDatafiles(tabletServer.getSystemConfiguration(), fs,
        extent, tabletsKeyValues), lookupTime(tabletServer.getSystemConfiguration(), extent,
tabletsKeyValues), lookupLastServer(extent, tabletsKeyValues),
        lookupScanFiles(extent, tabletsKeyValues, fs), lookupFlushID(extent, tabletsKeyValues),
lookupCompactID(extent, tabletsKeyValues));
  }
{code}

{{lookupDataFiles}} reaches back out to the metadata table to fetch this column. I'm not sure
if there's a reason why we can't do this all at once.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message