lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Sokolov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-3284) StreamingUpdateSolrServer swallows exceptions
Date Sun, 10 Jun 2012 12:15:42 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292507#comment-13292507
] 

Mike Sokolov edited comment on SOLR-3284 at 6/10/12 12:14 PM:
--------------------------------------------------------------

Another approach we have been using in a similar parallel writing situation is to have the
thread pool maintain a fixed-size list of exceptions.  Whenever a worker throws one, it gets
put on the list.  Then at certain barrier conditions (flush, exception list full), the exceptions
are collected together and re-thrown using an umbrella exception that wraps them.  At the
same time, all worker threads are terminated. You do need flush(), or can you rely on the
user calling commit() (and flush() internally then)?

This enables writing to continue even in the face of errors, but the errors do get reported
eventually.  This makes the system to be more robust in the face of transient failure conditions.

                
      was (Author: sokolov):
    Another approach we have been using in a similar parallel writing situation is to have
the thread pool maintain a fixed-size list of exceptions.  Whenever a worker throws one, it
gets put on the list.  Then at certain barrier conditions (flush, exception list full), the
exceptions are collected together and re-thrown using an umbrella exception that wraps them.
 You do need flush(), or can you rely on the user calling commit() (and flush() internally
then)?

This enables writing to continue even in the face of errors, but the errors do get reported
eventually.  This makes the system to be more robust in the face of transient failure conditions.

                  
> StreamingUpdateSolrServer swallows exceptions
> ---------------------------------------------
>
>                 Key: SOLR-3284
>                 URL: https://issues.apache.org/jira/browse/SOLR-3284
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java
>    Affects Versions: 3.5, 4.0
>            Reporter: Shawn Heisey
>         Attachments: SOLR-3284.patch
>
>
> StreamingUpdateSolrServer eats exceptions thrown by lower level code, such as HttpClient,
when doing adds.  It may happen with other methods, though I know that query and deleteByQuery
will throw exceptions.  I believe that this is a result of the queue/Runner design.  That's
what makes SUSS perform better, but it means you sacrifice the ability to programmatically
determine that there was a problem with your update.  All errors are logged via slf4j, but
that's not terribly helpful except with determining what went wrong after the fact.
> When using CommonsHttpSolrServer, I've been able to rely on getting an exception thrown
by pretty much any error, letting me use try/catch to detect problems.
> There's probably enough dependent code out there that it would not be a good idea to
change the design of SUSS, unless there were alternate constructors or additional methods
available to configure new/old behavior.  Fixing this is probably not trivial, so it's probably
a better idea to come up with a new server object based on CHSS.  This is outside my current
skillset.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message