Return-Path: Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: (qmail 59601 invoked from network); 10 Oct 2010 08:41:58 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Oct 2010 08:41:58 -0000 Received: (qmail 53663 invoked by uid 500); 10 Oct 2010 08:41:57 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 53568 invoked by uid 500); 10 Oct 2010 08:41:55 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 53561 invoked by uid 99); 10 Oct 2010 08:41:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 10 Oct 2010 08:41:55 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 10 Oct 2010 08:41:52 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o9A8fVa5026900 for ; Sun, 10 Oct 2010 08:41:31 GMT Message-ID: <32942915.64441286700091067.JavaMail.jira@thor> Date: Sun, 10 Oct 2010 04:41:31 -0400 (EDT) From: "tom liu (JIRA)" To: dev@lucene.apache.org Subject: [jira] Issue Comment Edited: (SOLR-1395) Integrate Katta MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/SOLR-1395?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D12919= 581#action_12919581 ]=20 tom liu edited comment on SOLR-1395 at 10/10/10 4:41 AM: --------------------------------------------------------- My deployment is: # one Master # two Slaves: #* solr01 #* solr02 # two Indexes: #* solrhome01(.zip) #* solrhome02(.zip) And, i use: {noformat} # bin/katta addIndex solrhome01 hdfs://localhost:9000/solr/solrhome01.zip # bin/katta addIndex solrhome02 hdfs://localhost:9000/solr/solrhome02.zip {noformat} so, my shard-Node is: # solrhome01#solrhome01 #* --solr01 #* --solr02 # solrhome02#solrhome02 #* --solr01 #* --solr02 When i searched in master, i found that, in any slave, the search ran twice= , such as: {noformat} SolrServer.request: solr01:20000 shards:[solrhome01#solrhome01, solrhome02#= solrhome02]=20 request params:fl=3Did%2Cscore&start=3D0&q=3Dsolr&isShard=3Dtrue&fsv=3Dtrue= &rows=3D10&shards=3Dsolrhome01%23solrhome01%2Csolrhome02%23solrhome02 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome02#solrhome02] webapp=3Dnull path=3D/select pa= rams=3D{fl=3Did%2Cscore&start=3D0&q=3Dsolr&isShard=3Dtrue&fsv=3Dtrue&rows= =3D10} hits=3D1 status=3D0 QTime=3D16=20 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome01#solrhome01] webapp=3Dnull path=3D/select pa= rams=3D{fl=3Did%2Cscore&start=3D0&q=3Dsolr&isShard=3Dtrue&fsv=3Dtrue&rows= =3D10} hits=3D1 status=3D0 QTime=3D16=20 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome02#solrhome02] webapp=3Dnull path=3D/select pa= rams=3D{fl=3Did%2Cscore%2Cid&start=3D0&q=3Dsolr&isShard=3Dtrue&rows=3D10&id= s=3DSOLR1000} status=3D0 QTime=3D0=20 SolrServer.SolrResponse:{response=3D{numFound=3D1,start=3D00.5747526,docs= =3D[SolrDocument[{id=3DSOLR1000, score=3D0.5747526}]]},QueriedShards=3D[Lja= va.lang.String;@175ace6} {noformat} {noformat} SolrServer.request: solr02:20000 shards:[solrhome01#solrhome01, solrhome02#= solrhome02]=20 request params:start=3D0&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&rows=3D10&s= hards=3Dsolrhome01%23solrhome01%2Csolrhome02%23solrhome02 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome02#solrhome02] webapp=3Dnull path=3D/select pa= rams=3D{start=3D0&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&rows=3D10&fsv=3Dtr= ue&fl=3Did%2Cscore} status=3D0 QTime=3D16 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome01#solrhome01] webapp=3Dnull path=3D/select pa= rams=3D{start=3D0&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&rows=3D10&fsv=3Dtr= ue&fl=3Did%2Cscore} status=3D0 QTime=3D16 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome01#solrhome01] webapp=3Dnull path=3D/select pa= rams=3D{start=3D0&ids=3DSOLR1000&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&row= s=3D10} status=3D0 QTime=3D0 SolrServer.SolrResponse:{response=3D{numFound=3D1,start=3D0,docs=3D[SolrDoc= ument[{id=3DSOLR1000, ...]]},QueriedShards=3D[Ljava.lang.String;@1d590d} {noformat} i think, in slaves, IS_SHARD=3Dtrue, so, it would prevent this happens. was (Author: tom_lt): My deployment is: # one Master # two Slaves: #* solr01 #* solr02 # two Indexes: #* solrhome01(.zip) #* solrhome02(.zip) And, i use: {noformat} # bin/katta addIndex solrhome01 hdfs://localhost:9000/solr/solrhome01.zip # bin/katta addIndex solrhome02 hdfs://localhost:9000/solr/solrhome02.zip {noformat} so, my shard-Node is: # solrhome01#solrhome01 #* --solr01 #* --solr02 # solrhome02#solrhome02 #* --solr01 #* --solr02 When i searched in master, i found that, in any slave, the search ran twice= , such as: {noformat} SolrServer.request: solr01:20000 shards:[solrhome01#solrhome01, solrhome02#= solrhome02]=20 request params:fl=3Did%2Cscore&start=3D0&q=3Dsolr&isShard=3Dtrue&fsv=3Dtrue= &rows=3D10&shards=3Dsolrhome01%23solrhome01%2Csolrhome02%23solrhome02 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome02#solrhome02] webapp=3Dnull path=3D/select pa= rams=3D{fl=3Did%2Cscore&start=3D0&q=3Dsolr&isShard=3Dtrue&fsv=3Dtrue&rows= =3D10} hits=3D1 status=3D0 QTime=3D16=20 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome01#solrhome01] webapp=3Dnull path=3D/select pa= rams=3D{fl=3Did%2Cscore&start=3D0&q=3Dsolr&isShard=3Dtrue&fsv=3Dtrue&rows= =3D10} hits=3D1 status=3D0 QTime=3D16=20 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome02#solrhome02] webapp=3Dnull path=3D/select pa= rams=3D{fl=3Did%2Cscore%2Cid&start=3D0&q=3Dsolr&isShard=3Dtrue&rows=3D10&id= s=3DSOLR1000} status=3D0 QTime=3D0=20 SolrServer.SolrResponse:{response=3D{numFound=3D1,start=3D00.5747526,docs= =3D[SolrDocument[{id=3DSOLR1000, score=3D0.5747526}]]},QueriedShards=3D[Lja= va.lang.String;@175ace6} {noformat} {noformat} SolrServer.request: tom-SL510:20001 shards:[solrhome01#solrhome01, solrhome= 02#solrhome02]=20 request params:start=3D0&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&rows=3D10&s= hards=3Dsolrhome01%23solrhome01%2Csolrhome02%23solrhome02 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome02#solrhome02] webapp=3Dnull path=3D/select pa= rams=3D{start=3D0&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&rows=3D10&fsv=3Dtr= ue&fl=3Did%2Cscore} status=3D0 QTime=3D16 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome01#solrhome01] webapp=3Dnull path=3D/select pa= rams=3D{start=3D0&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&rows=3D10&fsv=3Dtr= ue&fl=3Did%2Cscore} status=3D0 QTime=3D16 2010-10-10 16:17:04 org.apache.solr.core.SolrCore execute =E4=BF=A1=E6=81=AF: [solrhome01#solrhome01] webapp=3Dnull path=3D/select pa= rams=3D{start=3D0&ids=3DSOLR1000&ids=3DSOLR1000&q=3Dsolr&isShard=3Dtrue&row= s=3D10} status=3D0 QTime=3D0 SolrServer.SolrResponse:{response=3D{numFound=3D1,start=3D0,docs=3D[SolrDoc= ument[{id=3DSOLR1000, ...]]},QueriedShards=3D[Ljava.lang.String;@1d590d} {noformat} i think, in slaves, IS_SHARD=3Dtrue, so, it would prevent this happens. =20 > Integrate Katta > --------------- > > Key: SOLR-1395 > URL: https://issues.apache.org/jira/browse/SOLR-1395 > Project: Solr > Issue Type: New Feature > Affects Versions: 1.4 > Reporter: Jason Rutherglen > Priority: Minor > Fix For: Next > > Attachments: back-end.log, front-end.log, hadoop-core-0.19.0.jar,= katta-core-0.6-dev.jar, katta.node.properties, katta.zk.properties, log4j-= 1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, solr-1395-1431-= katta0.6.patch, solr-1395-1431-katta0.6.patch, solr-1395-1431.patch, SOLR-1= 395.patch, SOLR-1395.patch, SOLR-1395.patch, test-katta-core-0.6-dev.jar, z= kclient-0.1-dev.jar, zookeeper-3.2.1.jar > > Original Estimate: 336h > Remaining Estimate: 336h > > We'll integrate Katta into Solr so that: > * Distributed search uses Hadoop RPC > * Shard/SolrCore distribution and management > * Zookeeper based failover > * Indexes may be built using Hadoop --=20 This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org