phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chenglei (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-3850) Indexer.postOpen should not always log "Found some outstanding index updates that didn't succeed"
Date Sat, 13 May 2017 04:07:04 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

chenglei updated PHOENIX-3850:
------------------------------
    Description: 
When the RegionServer starts, I always find the region logs "Found some outstanding index
updates that didn't succeed during WAL replay - attempting to replay now." That is because
in following Indexer.postOpen method, the LOG.info in line 528 is before the if statement
in line 531, so the method always log "Found some outstanding index updates that didn't succeed..."
no matter whether there are outstanding index updates.

 {code}
520  @Override
521  public void postOpen(final ObserverContext<RegionCoprocessorEnvironment> c) {
522    Multimap<HTableInterfaceReference, Mutation> updates = failedIndexEdits.getEdits(c.getEnvironment().getRegion());
523    
524    if (this.disabled) {
525        super.postOpen(c);
526        return;
527      }
528   LOG.info("Found some outstanding index updates that didn't succeed during"
529        + " WAL replay - attempting to replay now.");
530    //if we have no pending edits to complete, then we are done
531    if (updates == null || updates.size() == 0) {
532      return;
533    }
534    
535    // do the usual writer stuff, killing the server again, if we can't manage to make
the index
536    // writes succeed again
537    try {
538        writer.writeAndKillYourselfOnFailure(updates, true);
539    } catch (IOException e) {
540            LOG.error("During WAL replay of outstanding index updates, "
541                    + "Exception is thrown instead of killing server during index writing",
e);
542    }
543  }
{code}

{code}
547   @Override
548   public void preWALRestore(ObserverContext<RegionCoprocessorEnvironment> env, HRegionInfo
info,
549       HLogKey logKey, WALEdit logEdit) throws IOException {
550       if (this.disabled) {
551          super.preWALRestore(env, info, logKey, logEdit);
552          return;
553        }
554    // TODO check the regions in transition. If the server on which the region lives is
this one,
555    // then we should rety that write later in postOpen.
556    // we might be able to get even smarter here and pre-split the edits that are server-local
557    // into their own recovered.edits file. This then lets us do a straightforward recovery
of each
558    // region (and more efficiently as we aren't writing quite as hectically from this
one place).
559
560    /*
561     * Basically, we let the index regions recover for a little while long before retrying
in the
562     * hopes they come up before the primary table finishes.
563     */
564    Collection<Pair<Mutation, byte[]>> indexUpdates = extractIndexUpdate(logEdit);
565    recoveryWriter.write(indexUpdates, true);
566  }
{code}


  was:
When the RegionServer starts, I always find the region logs "Found some outstanding index
updates that didn't succeed during WAL replay - attempting to replay now." That is because
in following Indexer.postOpen method, the LOG.info in line 528 is before the if statement
in line 531, so the method always log "Found some outstanding index updates that didn't succeed..."
no matter whether there are outstanding index updates.

 {code}
520  @Override
521  public void postOpen(final ObserverContext<RegionCoprocessorEnvironment> c) {
522    Multimap<HTableInterfaceReference, Mutation> updates = failedIndexEdits.getEdits(c.getEnvironment().getRegion());
523    
524    if (this.disabled) {
525        super.postOpen(c);
526        return;
527      }
528   LOG.info("Found some outstanding index updates that didn't succeed during"
529        + " WAL replay - attempting to replay now.");
530    //if we have no pending edits to complete, then we are done
531    if (updates == null || updates.size() == 0) {
532      return;
533    }
534    
535    // do the usual writer stuff, killing the server again, if we can't manage to make
the index
536    // writes succeed again
537    try {
538        writer.writeAndKillYourselfOnFailure(updates, true);
539    } catch (IOException e) {
540            LOG.error("During WAL replay of outstanding index updates, "
541                    + "Exception is thrown instead of killing server during index writing",
e);
542    }
543  }
{code}


547   @Override
548   public void preWALRestore(ObserverContext<RegionCoprocessorEnvironment> env, HRegionInfo
info,
549       HLogKey logKey, WALEdit logEdit) throws IOException {
550       if (this.disabled) {
551          super.preWALRestore(env, info, logKey, logEdit);
552          return;
553        }
554    // TODO check the regions in transition. If the server on which the region lives is
this one,
555    // then we should rety that write later in postOpen.
556    // we might be able to get even smarter here and pre-split the edits that are server-local
557    // into their own recovered.edits file. This then lets us do a straightforward recovery
of each
558    // region (and more efficiently as we aren't writing quite as hectically from this
one place).
559
560    /*
561     * Basically, we let the index regions recover for a little while long before retrying
in the
562     * hopes they come up before the primary table finishes.
563     */
564    Collection<Pair<Mutation, byte[]>> indexUpdates = extractIndexUpdate(logEdit);
565    recoveryWriter.write(indexUpdates, true);
566  }



> Indexer.postOpen should not always log "Found some outstanding index updates that didn't
succeed"
> -------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3850
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3850
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.10.0
>            Reporter: chenglei
>            Priority: Minor
>         Attachments: PHOENIX-3850_v1.patch
>
>
> When the RegionServer starts, I always find the region logs "Found some outstanding index
updates that didn't succeed during WAL replay - attempting to replay now." That is because
in following Indexer.postOpen method, the LOG.info in line 528 is before the if statement
in line 531, so the method always log "Found some outstanding index updates that didn't succeed..."
no matter whether there are outstanding index updates.
>  {code}
> 520  @Override
> 521  public void postOpen(final ObserverContext<RegionCoprocessorEnvironment> c)
{
> 522    Multimap<HTableInterfaceReference, Mutation> updates = failedIndexEdits.getEdits(c.getEnvironment().getRegion());
> 523    
> 524    if (this.disabled) {
> 525        super.postOpen(c);
> 526        return;
> 527      }
> 528   LOG.info("Found some outstanding index updates that didn't succeed during"
> 529        + " WAL replay - attempting to replay now.");
> 530    //if we have no pending edits to complete, then we are done
> 531    if (updates == null || updates.size() == 0) {
> 532      return;
> 533    }
> 534    
> 535    // do the usual writer stuff, killing the server again, if we can't manage to
make the index
> 536    // writes succeed again
> 537    try {
> 538        writer.writeAndKillYourselfOnFailure(updates, true);
> 539    } catch (IOException e) {
> 540            LOG.error("During WAL replay of outstanding index updates, "
> 541                    + "Exception is thrown instead of killing server during index
writing", e);
> 542    }
> 543  }
> {code}
> {code}
> 547   @Override
> 548   public void preWALRestore(ObserverContext<RegionCoprocessorEnvironment> env,
HRegionInfo info,
> 549       HLogKey logKey, WALEdit logEdit) throws IOException {
> 550       if (this.disabled) {
> 551          super.preWALRestore(env, info, logKey, logEdit);
> 552          return;
> 553        }
> 554    // TODO check the regions in transition. If the server on which the region lives
is this one,
> 555    // then we should rety that write later in postOpen.
> 556    // we might be able to get even smarter here and pre-split the edits that are
server-local
> 557    // into their own recovered.edits file. This then lets us do a straightforward
recovery of each
> 558    // region (and more efficiently as we aren't writing quite as hectically from
this one place).
> 559
> 560    /*
> 561     * Basically, we let the index regions recover for a little while long before
retrying in the
> 562     * hopes they come up before the primary table finishes.
> 563     */
> 564    Collection<Pair<Mutation, byte[]>> indexUpdates = extractIndexUpdate(logEdit);
> 565    recoveryWriter.write(indexUpdates, true);
> 566  }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message