lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wolfgang hoschek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-6126) MapReduce's GoLive script should support replicas
Date Mon, 02 Jun 2014 08:50:01 GMT

    [ https://issues.apache.org/jira/browse/SOLR-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015266#comment-14015266
] 

wolfgang hoschek commented on SOLR-6126:
----------------------------------------

[~dsmiley] It uses the --zk-host CLI options to fetch the solr URLs of each replica from zk
- see extractShardUrls(). This info gets passed via the Options.shardUrls parameter into the
go-live phase. In the go-live phase the segments of each shard are explicitly merged via a
separate REST merge request per replica into the corresponding replica. The result is that
each input segment is explicitly merged N times where N is the replication factor. Each such
merge reads from HDFS and writes to HDFS.

(BTW, I'll be unreachable on an transatlantic flight very soon)

> MapReduce's GoLive script should support replicas
> -------------------------------------------------
>
>                 Key: SOLR-6126
>                 URL: https://issues.apache.org/jira/browse/SOLR-6126
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - MapReduce
>            Reporter: David Smiley
>
> The GoLive feature of the MapReduce contrib module is pretty cool.  But a comment in
there indicates that it doesn't support replicas.  Every production SolrCloud setup I've seen
has had replicas!
> I wonder what is needed to support this.  For GoLive to work, it assumes a shared file
system (be it HDFS or whatever, like a SAN).  If perhaps the replicas in such a system read
from the very same network disk location, then all we'd need to do is send a commit() to replicas;
right?  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message