lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jessica Cheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks
Date Thu, 12 Dec 2013 23:05:08 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846868#comment-13846868
] 

Jessica Cheng commented on SOLR-5477:
-------------------------------------

Would you please comment more about how failures are handled? I'm interested especially in
how the Overseer may find out if individual subcommand to individual core admin API failed
vs. timed out (the same sort of problem on the overall collection task). Also, if it fails,
if and how states are cleaned up so that when the client re-issue the command it has a chance
of succeeding. (For example, if a split shard command fails for some unknown reason, it might
have left the shardX_0 and shardX_1 created, and the next split command might fail because
it tries to create those two new target shards but they already exist. Note that this is just
an example for explanation's sake--I don't know if it actually will complain this way.) Thanks!

> Async execution of OverseerCollectionProcessor tasks
> ----------------------------------------------------
>
>                 Key: SOLR-5477
>                 URL: https://issues.apache.org/jira/browse/SOLR-5477
>             Project: Solr
>          Issue Type: Sub-task
>          Components: SolrCloud
>            Reporter: Noble Paul
>            Assignee: Anshum Gupta
>
> Typical collection admin commands are long running and it is very common to have the
requests get timed out.  It is more of a problem if the cluster is very large.Add an option
to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id. 
> as separate collection admin command will be added to poll the status of the task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are completed
the queue entry is removed. OverSeerColectionProcessor will perform these tasks in multiple
threads



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message