accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3282) Reduce scans of metadata when assigning tablet
Date Thu, 13 Nov 2014 22:43:34 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211402#comment-14211402
] 

Eric Newton commented on ACCUMULO-3282:
---------------------------------------

I was hoping for something more comprehensive: a class that would model the data in the metadata
table.  Still, I'm sure this will accelerate initial tablet assignment.

> Reduce scans of metadata when assigning tablet
> ----------------------------------------------
>
>                 Key: ACCUMULO-3282
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3282
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.7.0
>
>         Attachments: ACCUMULO-3282.patch
>
>
> Noticed the following case digging through the assignment code:
> {code:title=TabletServer.java}
>   public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent extent,
TServerInstance instance, SortedMap<Key,Value> tabletsKeyValues,
>       String clientAddress, ZooLock lock) throws AccumuloSecurityException, DistributedStoreException,
AccumuloException {
> ...
>   public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent extent,
TServerInstance instance, SortedMap<Key,Value> tabletsKeyValues,
>       String clientAddress, ZooLock lock) throws AccumuloSecurityException, DistributedStoreException,
AccumuloException {
>     log.debug("verifying extent " + extent);
>     if (extent.isRootTablet()) {
>       return verifyRootTablet(extent, instance);
>     }
>     String tableToVerify = MetadataTable.ID;
>     if (extent.isMeta())
>       tableToVerify = RootTable.ID;
>     List<ColumnFQ> columnsToFetch = Arrays.asList(new ColumnFQ[] {TabletsSection.ServerColumnFamily.DIRECTORY_COLUMN,
>         TabletsSection.TabletColumnFamily.PREV_ROW_COLUMN, TabletsSection.TabletColumnFamily.SPLIT_RATIO_COLUMN,
>         TabletsSection.TabletColumnFamily.OLD_PREV_ROW_COLUMN, TabletsSection.ServerColumnFamily.TIME_COLUMN});
>     ScannerImpl scanner = new ScannerImpl(HdfsZooInstance.getInstance(), SystemCredentials.get(),
tableToVerify, Authorizations.EMPTY);
>     scanner.setRange(extent.toMetadataRange());
>     TreeMap<Key,Value> tkv = new TreeMap<Key,Value>();
>     for (Entry<Key,Value> entry : scanner)
>       tkv.put(entry.getKey(), entry.getValue());
>     // only populate map after success
>     if (tabletsKeyValues == null) {
>       tabletsKeyValues = tkv;
>     } else {
>       tabletsKeyValues.clear();
>       tabletsKeyValues.putAll(tkv);
> {code}
> Essentially, we read a few columns for the tablet's row from metadata and (when metadata
is in a consistent state) we return those columns to the Tablet constructor.
> {code:title=Tablet.java}
>   private Tablet(TabletServer tabletServer, Text location, KeyExtent extent, TabletResourceManager
trm, Configuration conf, VolumeManager fs,
>       SortedMap<Key,Value> tabletsKeyValues) throws IOException {
>     this(tabletServer, location, extent, trm, conf, fs, lookupLogEntries(extent, tabletsKeyValues),
lookupDatafiles(tabletServer.getSystemConfiguration(), fs,
>         extent, tabletsKeyValues), lookupTime(tabletServer.getSystemConfiguration(),
extent, tabletsKeyValues), lookupLastServer(extent, tabletsKeyValues),
>         lookupScanFiles(extent, tabletsKeyValues, fs), lookupFlushID(extent, tabletsKeyValues),
lookupCompactID(extent, tabletsKeyValues));
>   }
> {code}
> {{lookupDataFiles}} reaches back out to the metadata table to fetch this column. I'm
not sure if there's a reason why we can't do this all at once.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message