lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Campeau <cam...@gmail.com>
Subject Re: SolrCloud: Understanding Replication
Date Fri, 30 May 2014 17:38:33 GMT
2014-05-30 12:24 GMT-04:00 Erick Erickson <erickerickson@gmail.com>:

> Let's back up a bit here. Why are you copying your indexes around?
> SolrCloud does all this for you. I suspect you've somehow made a mis-step.
>

I started by copying the index around because my 4.5.1 instance is not
setup as Cloud and I wanted to avoid reindexing all my data when migrating
to my new 4.8.1 SolrCloud setup. I've now put that aside and I'm just
trying to get replication happening when I populate an empty collection.


> So here's what I'd do by preference; Just set up a new collection and
> re-index. Make sure all of the nodes are up and then just go ahead and
> index to any of them. If you're using SolrJ, CloudSolrServer will be a bit
> more efficient than sending the docs to random nodes, but that's not
> necessary.
>

I've been trying that this morning. Stop the instances, deleted the
contents of  /data on all my 4.8.1 instances then started them again...
they all show up in a 1 shard cluster as 4 replicas and one is the
leader... they're still shown as down in clusterstate. Then I sent a
document to be added to one of the nodes specifically. Only that node now
contains the document. It hasn't been replicated to the other instances.

When I issue queries to the collection for that document through my load
balancer it works roughtly 1/4 times, in accordance with the fact that it's
only on the instance where it was added.

Must I use the CLI API for collections to create this new collection or can
I just do it old style by creating subfolder in /solr directory with my
confs?


Here's the log of these operations

LOG of Instance where document was added:
--------------------
2758138 [qtp1781256139-14] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
{add=[Listing_3446279]} 0 271
2769177 [qtp1781256139-12] INFO  org.apache.solr.core.SolrCore  –
[mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
QTime=1

[... More Pings ... ]

2773138 [commitScheduler-7-thread-1] INFO
 org.apache.solr.update.UpdateHandler  – start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
2773377 [commitScheduler-7-thread-1] INFO
 org.apache.solr.search.SolrIndexSearcher  – Opening
Searcher@175816a5[mycollection]
main
2773389 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore
 – QuerySenderListener sending requests to Searcher@175816a5[mycollection]
main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
2773389 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore
 – QuerySenderListener done.
2773390 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore
 – [mycollection] Registered new searcher Searcher@175816a5[mycollection]
main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
2773390 [commitScheduler-7-thread-1] INFO
 org.apache.solr.update.UpdateHandler  – end_commit_flush

[... More Pings ... ]

2799792 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2799883 [qtp1781256139-18] INFO  org.apache.solr.core.SolrCore  –
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(MMapDirectory@/opt/solr-4.8.0/example/solr/mycollection/data/index
lockFactory=NativeFSLockFactory@/opt/solr-4.8.0/example/solr/mycollection/data/index;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
commit{dir=NRTCachingDirectory(MMapDirectory@/opt/solr-4.8.0/example/solr/mycollection/data/index
lockFactory=NativeFSLockFactory@/opt/solr-4.8.0/example/solr/mycollection/data/index;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
2799884 [qtp1781256139-18] INFO  org.apache.solr.core.SolrCore  – newest
commit generation = 2
2799887 [qtp1781256139-18] INFO  org.apache.solr.core.SolrCore  –
SolrIndexSearcher has not changed - not re-opening:
org.apache.solr.search.SolrIndexSearcher
2799887 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
end_commit_flush
2799888 [qtp1781256139-18] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
{commit=} 0 96
2800051 [qtp1781256139-14] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {commit=}
0 611
2802085 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2802085 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  – No
uncommitted changes. Skipping IW.commit.
2802086 [qtp1781256139-18] INFO  org.apache.solr.core.SolrCore  –
SolrIndexSearcher has not changed - not re-opening:
org.apache.solr.search.SolrIndexSearcher
2802086 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
end_commit_flush
2802087 [qtp1781256139-18] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
{commit=} 0 2
2802126 [qtp1781256139-14] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {commit=}
0 44

[... More Pings ... ]
----------------------

LOG of 3 other instances :
----------------------
2773666 [qtp1781256139-11] INFO  org.apache.solr.core.SolrCore  –
[mycollection] webapp=/solr path=/admin/ping params={} status=0 QTime=2
2774531 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2774531 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  – No
uncommitted changes. Skipping IW.commit.
2774532 [qtp1781256139-18] INFO  org.apache.solr.core.SolrCore  –
SolrIndexSearcher has not changed - not re-opening:
org.apache.solr.search.SolrIndexSearcher
2774533 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
end_commit_flush
2774533 [qtp1781256139-18] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
{commit=} 0 23
2776603 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2776603 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  – No
uncommitted changes. Skipping IW.commit.
2776604 [qtp1781256139-18] INFO  org.apache.solr.core.SolrCore  –
SolrIndexSearcher has not changed - not re-opening:
org.apache.solr.search.SolrIndexSearcher
2776604 [qtp1781256139-18] INFO  org.apache.solr.update.UpdateHandler  –
end_commit_flush
2776604 [qtp1781256139-18] INFO
 org.apache.solr.update.processor.LogUpdateProcessor  – [mycollection]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
{commit=} 0 2
2777428 [qtp1781256139-12] INFO  org.apache.solr.core.SolrCore  –
[mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
QTime=0
----------------------


> If that isn't feasible, set up a _one_ node "cloud" and get that running
> and showing as up with your current index. Then use the Collections
> "ADDREPLICA" command to bring up the other three nodes. All the index
> synching should then "just happen".
>

Will try that.


> You're also confusing master/slave replication with SolrCloud. In the
> normal state, there is no use of the older-style replication, except when a
> downed or new node comes online. If "peer synch" isn't possible, then old
> style replication happens, but that should be the only time it's used.
> Outside those edge cases, updates go to all nodes when sent to any node,
> otherwise NRT wouldn't work.
>

That's what I understood from the research I made. I just think that I
missed some config or something that will make replication happen.

Thanks for your time,

Marc

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message