lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amrit Sarkar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-12057) CDCR does not replicate to Collections with TLOG Replicas
Date Thu, 25 Oct 2018 20:00:00 GMT

     [ https://issues.apache.org/jira/browse/SOLR-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amrit Sarkar updated SOLR-12057:
--------------------------------
    Attachment:     (was: SOLR-12057.patch)

> CDCR does not replicate to Collections with TLOG Replicas
> ---------------------------------------------------------
>
>                 Key: SOLR-12057
>                 URL: https://issues.apache.org/jira/browse/SOLR-12057
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: CDCR
>    Affects Versions: 7.2
>            Reporter: Webster Homer
>            Assignee: Varun Thacker
>            Priority: Major
>         Attachments: SOLR-12057.patch, SOLR-12057.patch, SOLR-12057.patch, SOLR-12057.patch,
SOLR-12057.patch, cdcr-fail-with-tlog-pull.patch, cdcr-fail-with-tlog-pull.patch
>
>
> We created a collection using TLOG replicas in our QA clouds.
> We have a locally hosted solrcloud with 2 nodes, all our collections have 2 shards. We
use CDCR to replicate the collections from this environment to 2 data centers hosted in Google
cloud. This seems to work fairly well for our collections with NRT replicas. However the new
TLOG collection has problems.
>  
> The google cloud solrclusters have 4 nodes each (3 separate Zookeepers). 2 shards per
collection with 2 replicas per shard.
>  
> We never see data show up in the cloud collections, but we do see tlog files show up
on the cloud servers. I can see that all of the servers have cdcr started, buffers are disabled.
> The cdcr source configuration is:
>  
> "requestHandler":{"/cdcr":{
>       "name":"/cdcr",
>       "class":"solr.CdcrRequestHandler",
>       "replica":[
>         {
>           "zkHost":"[xxx-mzk01.sial.com:2181|http://xxx-mzk01.sial.com:2181/],[xxx-mzk02.sial.com:2181|http://xxx-mzk02.sial.com:2181/],[xxx-mzk03.sial.com:2181/solr|http://xxx-mzk03.sial.com:2181/solr]",
>           "source":"b2b-catalog-material-180124T",
>           "target":"b2b-catalog-material-180124T"},
>         {
>           "zkHost":"[yyyy-mzk01.sial.com:2181|http://yyyy-mzk01.sial.com:2181/],[yyyy-mzk02.sial.com:2181|http://yyyy-mzk02.sial.com:2181/],[yyyy-mzk03.sial.com:2181/solr|http://yyyy-mzk03.sial.com:2181/solr]",
>           "source":"b2b-catalog-material-180124T",
>           "target":"b2b-catalog-material-180124T"}],
>       "replicator":{
>         "threadPoolSize":4,
>         "schedule":500,
>         "batchSize":250},
>       "updateLogSynchronizer":\{"schedule":60000}}}}
>  
> The target configurations in the 2 clouds are the same:
> "requestHandler":{"/cdcr":{ "name":"/cdcr", "class":"solr.CdcrRequestHandler", "buffer":{"defaultState":"disabled"}}} 
>  
> All of our collections have a timestamp field, index_date. In the source collection all
the records have a date of 2/28/2018 but the target collections have a latest date of 1/26/2018
>  
> I don't see cdcr errors in the logs, but we use logstash to search them, and we're still
perfecting that. 
>  
> We have a number of similar collections that behave correctly. This is the only collection
that is a TLOG collection. It appears that CDCR doesn't support TLOG collections.
>  
> It looks like the data is getting to the target servers. I see tlog files with the right
timestamps. Looking at the timestamps on the documents in the collection none of the data
appears to have been loaded.In the solr.log I see lots of /cdcr messages  action=LASTPROCESSEDVERSION,
 action=COLLECTIONCHECKPOINT, and  action=SHARDCHECKPOINT 
>  
> no errors
>  
> Target collections autoCommit is set to  60000 I tried sending a commit explicitly no
difference. cdcr is uploading data, but no new data appears in the collection.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message