lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Traynham (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (SOLR-2326) Replication command indexversion fails to return index version
Date Mon, 09 Jan 2012 20:35:39 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182775#comment-13182775
] 

Matt Traynham edited comment on SOLR-2326 at 1/9/12 8:34 PM:
-------------------------------------------------------------

Hey Yury, I recently started seeing this same issue and thought I'd provide a bit of input
into what I found debugging in my 3.3 Branch. 
I have found that a core reload does break the commits call after.  But if you actually reload
a second time, it is fixed again.  This is because it is forcefully opening a new writer and
a lock exception occurs every other time.

During the inform method of ReplicationHandler, if you have configured replicate after startup,
the direct update handler will forceOpenWriter().

{code:title=ReplicationHandler.java}
if (replicateAfter.contains("startup")) {
                replicateOnStart = true;
                RefCounted<SolrIndexSearcher> s = core.getNewestSearcher(false);
                try {
                    IndexReader reader = s==null ? null : s.get().getReader();
                    if (reader!=null && reader.getIndexCommit() != null &&
reader.getIndexCommit().getGeneration() != 1L) {
                        try {
                            if(replicateOnOptimize){
                                Collection<IndexCommit> commits = IndexReader.listCommits(reader.directory());
                                for (IndexCommit ic : commits) {
                                    if(ic.isOptimized()){
                                        if(indexCommitPoint == null || indexCommitPoint.getVersion()
< ic.getVersion()) indexCommitPoint = ic;
                                    }
                                }
                            } else{
                                indexCommitPoint = reader.getIndexCommit();
                            }
                        } finally {
                            // We don't need to save commit points for replication, the SolrDeletionPolicy
                            // always saves the last commit point (and the last optimized
commit point, if needed)
                            /***
                            if(indexCommitPoint != null){
                                core.getDeletionPolicy().saveCommitPoint(indexCommitPoint.getVersion());
                             }
                             ***/
                        }
                    }
                    if (core.getUpdateHandler() instanceof DirectUpdateHandler2) {
                        ((DirectUpdateHandler2) core.getUpdateHandler()).forceOpenWriter();
                    } else {
                        LOG.warn("The update handler being used is not an instance or sub-class
of DirectUpdateHandler2. " +
                                "Replicate on Startup cannot work.");
                    } 
{code}

Which will request a new lock, open a new writer and unlock.  If a lock already exists the
exception will be thrown:
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out and actually bail
out of creating a new writer.

{code:title=DirectUpdateHandler2.java}
   public void forceOpenWriter() throws IOException  {
    iwCommit.lock();
    try {
      openWriter();
    } finally {
      iwCommit.unlock();
    }
  }
{code}

The openWriter method goes on to create a new SolrIndexWriter as well as a few other objects
like IndexFileDeleter and IndexDeletionPolicyWrapper, which actually holds the commitPoints.

{code:title=IndexDeletionPolicyWrapper.java}
private volatile Map<Long, IndexCommit> solrVersionVsCommits = new ConcurrentHashMap<Long,
IndexCommit>();

  /**
   * Internal use for Lucene... do not explicitly call.
   */
  public void onInit(List list) throws IOException {
    List<IndexCommitWrapper> wrapperList = wrap(list);
    deletionPolicy.onInit(wrapperList);
    updateCommitPoints(wrapperList);
    cleanReserves();
  }

  private void updateCommitPoints(List<IndexCommitWrapper> list) {
    Map<Long, IndexCommit> map = new ConcurrentHashMap<Long, IndexCommit>();
    for (IndexCommitWrapper wrapper : list) {
      if (!wrapper.isDeleted())
        map.put(wrapper.getVersion(), wrapper.delegate);
    }
    solrVersionVsCommits = map;
    latestCommit = ((list.get(list.size() - 1)).delegate);
  }

  /**
   * Gets the commit points for the index.
   * This map instance may change between commits and commit points may be deleted.
   * It is recommended to reserve a commit point for the duration of usage
   *
   * @return a Map of version to commit points
   */
  public Map<Long, IndexCommit> getCommits() {
    return solrVersionVsCommits;
  }
{code}

The problem being, if a writer never gets created correctly, the init method on the IndexDeletionPolicyWrapper
never gets called and the solrVersionVsCommits map is empty.  If anyone has any input on a
solution, that would be greatly appreciated.

Thanks,
Matt
                
      was (Author: mtraynham):
    Hey Yury, I recently started seeing this same issue and thought I'd provide a bit of input
into what I found debugging in my 3.3 Branch. 
I have found that a core reload does break the commits call after.  But if you actually reload
a second time, it is fixed again.  This is because it is forcefully opening a new writer and
a lock exception occurs every other time.

During the inform method of ReplicationHandler, if you have configured replicate after startup,
the direct update handler will forceOpenWriter().

{code:title=ReplicationHandler.java}
if (replicateAfter.contains("startup")) {
                replicateOnStart = true;
                RefCounted<SolrIndexSearcher> s = core.getNewestSearcher(false);
                try {
                    IndexReader reader = s==null ? null : s.get().getReader();
                    if (reader!=null && reader.getIndexCommit() != null &&
reader.getIndexCommit().getGeneration() != 1L) {
                        try {
                            if(replicateOnOptimize){
                                Collection<IndexCommit> commits = IndexReader.listCommits(reader.directory());
                                for (IndexCommit ic : commits) {
                                    if(ic.isOptimized()){
                                        if(indexCommitPoint == null || indexCommitPoint.getVersion()
< ic.getVersion()) indexCommitPoint = ic;
                                    }
                                }
                            } else{
                                indexCommitPoint = reader.getIndexCommit();
                            }
                        } finally {
                            // We don't need to save commit points for replication, the SolrDeletionPolicy
                            // always saves the last commit point (and the last optimized
commit point, if needed)
                            /***
                            if(indexCommitPoint != null){
                                core.getDeletionPolicy().saveCommitPoint(indexCommitPoint.getVersion());
                             }
                             ***/
                        }
                    }
                    if (core.getUpdateHandler() instanceof DirectUpdateHandler2) {
                        ((DirectUpdateHandler2) core.getUpdateHandler()).forceOpenWriter();
                    } else {
                        LOG.warn("The update handler being used is not an instance or sub-class
of DirectUpdateHandler2. " +
                                "Replicate on Startup cannot work.");
                    } 
{code}

Which will request a new lock, open a new writer and unlock.  If a lock already exists the
exception will be thrown:
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out and actually bail
out of creating a new writer.

{code:title=DirectUpdateHandler2.java}
   public void forceOpenWriter() throws IOException  {
    iwCommit.lock();
    try {
      openWriter();
    } finally {
      iwCommit.unlock();
    }
  }
{code}

The openWriter method goes on to create a new SolrIndexWriter as well as a few other objects
like IndexFileDeleter and IndexDeletionPolicyWrapper, which actually holds the commitPoints.

{code:title=IndexDeletionPolicyWrapper}
private volatile Map<Long, IndexCommit> solrVersionVsCommits = new ConcurrentHashMap<Long,
IndexCommit>();

  /**
   * Internal use for Lucene... do not explicitly call.
   */
  public void onInit(List list) throws IOException {
    List<IndexCommitWrapper> wrapperList = wrap(list);
    deletionPolicy.onInit(wrapperList);
    updateCommitPoints(wrapperList);
    cleanReserves();
  }

  private void updateCommitPoints(List<IndexCommitWrapper> list) {
    Map<Long, IndexCommit> map = new ConcurrentHashMap<Long, IndexCommit>();
    for (IndexCommitWrapper wrapper : list) {
      if (!wrapper.isDeleted())
        map.put(wrapper.getVersion(), wrapper.delegate);
    }
    solrVersionVsCommits = map;
    latestCommit = ((list.get(list.size() - 1)).delegate);
  }

  /**
   * Gets the commit points for the index.
   * This map instance may change between commits and commit points may be deleted.
   * It is recommended to reserve a commit point for the duration of usage
   *
   * @return a Map of version to commit points
   */
  public Map<Long, IndexCommit> getCommits() {
    return solrVersionVsCommits;
  }
{code}

The problem being, if a writer never gets created correctly, the init method on the IndexDeletionPolicyWrapper
never gets called and the solrVersionVsCommits map is empty.  If anyone has any input on a
solution, that would be greatly appreciated.

Thanks,
Matt
                  
> Replication command indexversion fails to return index version
> --------------------------------------------------------------
>
>                 Key: SOLR-2326
>                 URL: https://issues.apache.org/jira/browse/SOLR-2326
>             Project: Solr
>          Issue Type: Bug
>          Components: replication (java)
>         Environment: Branch 3x latest
>            Reporter: Eric Pugh
>            Assignee: Mark Miller
>             Fix For: 3.6, 4.0
>
>
> To test this, I took the /example/multicore/core0 solrconfig and added a simple replication
handler:
>   <requestHandler name="/replication" class="solr.ReplicationHandler" >
>       <lst name="master">
>         <str name="replicateAfter">commit</str>
>         <str name="replicateAfter">startup</str>
>         <str name="confFiles">schema.xml</str>
>       </lst>
>   </requestHandler>
> When I query the handler for details I get back the indexVersion that I expect: http://localhost:8983/solr/core0/replication?command=details&wt=json&indent=true
> But when I ask for just the indexVersion I get back a 0, which prevent the slaves from
pulling updates: http://localhost:8983/solr/core0/replication?command=indexversion&wt=json&indent=true

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message