phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 程磊 <comnetw...@163.com>
Subject Why Index.preWALRestore method just use RecoveryIndexWriter.write method to write the indexUpdates? It is a bug?
Date Sat, 13 May 2017 04:02:27 GMT
When I look the code of Index.preWALRestore method, I find RecoveryIndexWriter.write method
is used to write the indexUpdates in following line 565:


547 @Override
548 public void preWALRestore(ObserverContext<RegionCoprocessorEnvironment> env, HRegionInfo
info, 549 HLogKey logKey, WALEdit logEdit) throws IOException { 550 if (this.disabled) { 551
super.preWALRestore(env, info, logKey, logEdit); 552 return; 553 } 554 // TODO check the regions
in transition. If the server on which the region lives is this one, 555 // then we should
rety that write later in postOpen. 556 // we might be able to get even smarter here and pre-split
the edits that are server-local 557 // into their own recovered.edits file. This then lets
us do a straightforward recovery of each 558 // region (and more efficiently as we aren't
writing quite as hectically from this one place). 559 560 /* 561 * Basically, we let the index
regions recover for a little while long before retrying in the 562 * hopes they come up before
the primary table finishes. 563 */ 564 Collection<Pair<Mutation, byte[]>> indexUpdates
= extractIndexUpdate(logEdit); 565 recoveryWriter.write(indexUpdates, true); 566 }




but the RecoveryIndexWriter.write method is as following, it directly throws Exception except
non-existing tables, so RecoveryIndexWriter's failurePolicy(which is StoreFailuresInCachePolicy
by default) even has no opportunity to be used, and it leads to Index.failedIndexEdits which
is filled by the StoreFailuresInCachePolicy is always empty.


@Override public void write(Collection<Pair<Mutation, byte[]>> toWrite, boolean
allowLocalUpdates) throws IOException { try { write(resolveTableReferences(toWrite), allowLocalUpdates);
} catch (MultiIndexWriteFailureException e) { for (HTableInterfaceReference table : e.getFailedTables())
{ if (!admin.tableExists(table.getTableName())) { LOG.warn("Failure due to non existing table:
" + table.getTableName()); nonExistingTablesList.add(table); } else { throw e; } } } }


So the Index.postOpen method seems useless,because the updates Multimap in following 522 line
which is geted from Index.failedIndexEdits is always empty.


I think in Index.preWALRestore method, we should use RecoveryWriter.writeAndKillYourselfOnFailure
method to write the indexUpdates, not the RecoveryIndexWriter.write method.






520  @Override
521  public void postOpen(final ObserverContext<RegionCoprocessorEnvironment> c) {
522    Multimap<HTableInterfaceReference, Mutation> updates = failedIndexEdits.getEdits(c.getEnvironment().getRegion());
523    
524    if (this.disabled) {
525        super.postOpen(c);
526        return;
527      }
528   LOG.info("Found some outstanding index updates that didn't succeed during"
529        + " WAL replay - attempting to replay now.");
530    //if we have no pending edits to complete, then we are done
531    if (updates == null || updates.size() == 0) {
532      return;
533    }
534    
535    // do the usual writer stuff, killing the server again, if we can't manage to make
the index
536    // writes succeed again
537    try {
538        writer.writeAndKillYourselfOnFailure(updates, true);
539    } catch (IOException e) {
540            LOG.error("During WAL replay of outstanding index updates, "
541                    + "Exception is thrown instead of killing server during index writing",
e);
542    }
543  }



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message