lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Thacker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-11326) CDCR bootstrap should not download tlog's from source
Date Tue, 05 Sep 2017 20:52:00 GMT

     [ https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Varun Thacker updated SOLR-11326:
---------------------------------
    Attachment: SOLR-11326.patch

Patch which ensures that tlog's won't be downloaded if the caller doesn't want to . All tests
pass

> CDCR bootstrap should not download tlog's from source
> -----------------------------------------------------
>
>                 Key: SOLR-11326
>                 URL: https://issues.apache.org/jira/browse/SOLR-11326
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Varun Thacker
>         Attachments: SOLR-11326.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap the tlog's
from the source is getting download
> snippet1:
> {code}
>    [junit4]   2> 42931 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr c:cdcr-source
s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Submitting
bootstrap task to executor
>    [junit4]   2> 42934 INFO  (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr
x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:53178_solr
c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager
Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
>    [junit4]   2> 43003 INFO  (qtp1525032019-69) [n:127.0.0.1:53178_solr c:cdcr-source
s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1]
 webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion}
status=0 QTime=0
>    [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr
c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's
generation: 12
>    [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr
c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's
version: 1503514968639
>    [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr
c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's
generation: 1
>    [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr
c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's
version: 0
>    [junit4]   2> 43004 INFO  (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr
x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr
c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting
replication process
>    [junit4]   2> 43041 INFO  (qtp1525032019-71) [n:127.0.0.1:53178_solr c:cdcr-source
s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Adding tlog
files to list: [{size=4649, name=tlog.0000000000000000000.1576549701811961856}, {size=4770,
name=tlog.0000000000000000001.1576549702515556352}, {size=4770, name=tlog.0000000000000000002.1576549702628802560},
{size=4770, name=tlog.0000000000000000003.1576549702720028672}, {size=4770, name=tlog.0000000000000000004.1576549702799720448},
{size=4770, name=tlog.0000000000000000005.1576549702894092288}, {size=4770, name=tlog.0000000000000000006.1576549703029358592},
{size=4770, name=tlog.0000000000000000007.1576549703126876160}, {size=4770, name=tlog.0000000000000000008.1576549703208665088},
{size=4770, name=tlog.0000000000000000009.1576549703295696896}
> {code}
> snippet2:
> {code}
>  17070    [junit4]   2> 677606 INFO  (qtp22544544-5725) [    ] o.a.s.h.CdcrReplicatorManager
Attempting to bootstrap target collection: cdcr-target, shard: shard1^M
>  17071    [junit4]   2> 677608 INFO  (qtp22544544-5725) [    ] o.a.s.h.CdcrReplicatorManager
Submitting bootstrap task to executor^M
> 17091    [junit4]   2> 677627 INFO  (qtp22544544-5724) [    ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1]
 webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion}
status=0 QTime=0^M
>  17092    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Master's generation: 12^M
>  17093    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Master's version: 1504127155366^M
>  17094    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Slave's generation: 1^M
>  17095    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Slave's version: 0^M
>  17096    [junit4]   2> 677628 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Starting replication process^M
>  17097    [junit4]   2> 677654 INFO  (qtp22544544-5725) [    ] o.a.s.h.ReplicationHandler
Adding tlog files to list: [{size=4773, name=tlog.0000000000000000009}]^M
>  17098    [junit4]   2> 677655 INFO  (qtp22544544-5725) [    ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1]
 webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist}
status=0 QTime=26^M
>  17099    [junit4]   2> 677656 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Number of files in latest index in master: 23^M
>  17100    [junit4]   2> 677656 INFO  (recoveryExecutor-1024-thread-1) [    ] o.a.s.h.IndexFetcher
Number of tlog files in master: 1^M
> {code}
> Also here's the code comment which also says we don't need to download the tlog's
> {code:title=CdcrRequestHandler.java|borderStyle=solid}
>         ModifiableSolrParams solrParams = new ModifiableSolrParams();
>         solrParams.set(ReplicationHandler.MASTER_URL, masterUrl);
>         // we do not want the raw tlog files from the source
>         solrParams.set(ReplicationHandler.TLOG_FILES, false);
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message