Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1E78111F85 for ; Tue, 15 Jul 2014 22:49:06 +0000 (UTC) Received: (qmail 45968 invoked by uid 500); 15 Jul 2014 22:49:04 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 45897 invoked by uid 500); 15 Jul 2014 22:49:04 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 45886 invoked by uid 99); 15 Jul 2014 22:49:04 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Jul 2014 22:49:04 +0000 Date: Tue, 15 Jul 2014 22:49:04 +0000 (UTC) From: "Nathan Neulinger (JIRA)" To: dev@lucene.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SOLR-6251) incorrect 'missing required field' during update - document definitely has it MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SOLR-6251?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D14062= 793#comment-14062793 ]=20 Nathan Neulinger commented on SOLR-6251: ---------------------------------------- 16.24 =3D POD SRV 16.204 =3D SOLR 1 16.207 =3D SOLR 2 16.24 =E2=87=92 16.204=20 CAP 1 11344 14:29:49.299883 POST /solr/d-_v22/update/json?commit=3Dtrue HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/json; charset=3DUTF-8 request_id: null 8677c2fb-8b92-4220-bb73-1e4c610d95be 2057 User-Agent: HivePoint (Factory JSON client:null:2056) X-Forwarded-For: 10.220.16.229 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 1555 Connection: keep-alive { "add": { "commitWithin" : 5000, "doc" : {"hive":"vdates","at":"2014-07-10= T21:28:41Z","timestamp":1405027721000,"type":"MESSAGE","channel":["dev"],"f= rom":"preet@sevogle.com","to":["adam@sevogle.com","vidya@sevogle.com","dev@= sevogle.com","sqa@hive.sevogle.com"],"subject":"Re: Deployments - B and the= n C","body":"eve.....SNIP.......stem. ","id":"4b2c4d09-31e2-4fe2-b767-3868e= fbdcda1","message_id":"2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1"} } = } 16.204 =E2=87=92 16.207 CAP 1 POST /solr/d-_v22_shard1_replica2/update?update.distrib=3DTOLEADER&distrib.= from=3Dhttp%3A%2F%2F10.220.16.204%3A8983%2Fsolr%2Fd-_v22_shard1_replica1%2F= &wt=3Djavabin&version=3D2 HTTP/1.1 User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrServer] 1.0 Content-Type: application/javabin Transfer-Encoding: chunked Host: 10.220.16.207:8983 Connection: Keep-Alive 64c ...¶ms...update.distrib(TOLEADER.,distrib.from?.http://10.220.16.204:89= 83/solr/d-_v22_shard1_replica1/.&delByQ..'docsMap.....?....$hive&vdates."at= 42014-07-10T21:28:41Z.)timestampx.......$type'MESSAGE.'channel.#dev.$from1p= reet@sevogle.com."to.0adam@sevogle.com1vidya@sevogle.com/dev@sevogle.com4sq= a@hive.sevogle.com.'subject>Re: Deployments - B and then C.$body?#eve.....S= NIP.......tem. ."id?.4b2c4d09-31e2-4fe2-b767-3868efbdcda1.*message_id?.2014= -07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1 .."ow.."cwX... 0 16.207 =E2=87=92 16.204=20 CAP 1 11368 14:29:49.495301 HTTP/1.1 200 OK Content-Type: application/octet-stream Content-Length: 40 ....responseHeader..&status......%QTimeK 16.24 =E2=87=92 16.204=20 CAP 1 11371 14:29:49.496308 INDEX COMPLETE HTTP/1.1 200 OK Content-Type: text/plain;charset=3DUTF-8 Transfer-Encoding: chunked 2C {"responseHeader":{"status":0,"QTime":195}} 0 16.24 =E2=87=92 16.207=20 CAP 2 9218 14:29:57.065156 9232 14:29=E2=80=9D57.099274 Search (two different search results to two servers?) that show the timesta= mp is set. POST /solr/d-_v22/select?indent=3Don&wt=3Djson HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/x-www-form-urlencoded; charset=3DUTF-8 request_id: null 957d1ca5-7200-4058-9c70-16a17fc64c19 2069 User-Agent: HivePoint (Factory JSON client:null:2068) X-Forwarded-For: 10.220.16.229 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 244 Connection: keep-alive q=3D%2B%28*%29&fq=3D%2Bhive%3Avdates+AND+%2Bchannel%3A%28adam+bethany+dev+n= otifications+preet+share%29+AND+at%3A%5B2014-07-10T21%3A27%3A56Z+TO+*%5D&st= art=3D0&rows=3D300&sort=3Dat+desc%2C+id+desc&fl=3Did,hive,timestamp,type,me= ssage_id,file_instance_id,scoreHTTP/1.1 200 OK Content-Type: text/plain;charset=3DUTF-8 Transfer-Encoding: chunked 2BB { "responseHeader":{ "status":0, "QTime":3, "params":{ "fl":"id,hive,timestamp,type,message_id,file_instance_id,score", "sort":"at desc, id desc", "indent":"on", "start":"0", "q":"+(*)", "wt":"json", "fq":"+hive:vdates AND +channel:(adam bethany dev notifications preet= share) AND at:[2014-07-10T21:27:56Z TO *]", "rows":"300"}}, "response":{"numFound":1,"start":0,"maxScore":1.0,"docs":[ { "hive":"vdates", "timestamp":1405027721000, "type":"MESSAGE", "id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1", "message_id":"2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1", "score":1.0}] }} 0 16.24 =E2=87=92 16.207=20 CAP 2 9415 14:30:00.310995 Update Channel POST /solr/d-_v22/update?commit=3Dtrue HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/json; charset=3DUTF-8 request_id: null 92fa6c11-78d8-44cc-a143-9ff3e4c132f4 2115 User-Agent: HivePoint (Factory JSON client:null:2114) X-Forwarded-For: 10.220.16.229 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 102 Connection: keep-alive [{"id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1","channel": {"add": "preet"},"= channel": {"add": "adam"}}]HTTP/1.1 400 Bad Request Content-Type: text/plain;charset=3DUTF-8 Transfer-Encoding: chunked 96 {"responseHeader":{"status":400,"QTime":1},"error":{"msg":"[doc=3D4b2c4d09-= 31e2-4fe2-b767-3868efbdcda1] missing required field: timestamp","code":400}= } 0 CAP 2 9602 14:30:08.082758 Subsequent search, after update POST /solr/d-_v22/select?indent=3Don&wt=3Djson HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/x-www-form-urlencoded; charset=3DUTF-8 request_id: null 196bee69-e79c-455e-b0cb-6ad6ecdab4e0 1813 User-Agent: HivePoint (Factory JSON client:null:1811) X-Forwarded-For: 10.220.16.230 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 261 Connection: keep-alive q=3D%2B%28*+-%28%28foo%29%29%29&fq=3D%2Bhive%3Avdates+AND+%2Bchannel%3A%28a= dam+bethany+dev+notifications+preet+share%29+AND+at%3A%5B2014-07-10T21%3A28= %3A07Z+TO+*%5D&start=3D0&rows=3D300&sort=3Dat+desc%2C+id+desc&fl=3Did,hive,= timestamp,type,message_id,file_instance_id,scoreHTTP/1.1 200 OK Content-Type: text/plain;charset=3DUTF-8 Transfer-Encoding: chunked 2C4 { "responseHeader":{ "status":0, "QTime":3, "params":{ "fl":"id,hive,timestamp,type,message_id,file_instance_id,score", "sort":"at desc, id desc", "indent":"on", "start":"0", "q":"+(* -((foo)))", "wt":"json", "fq":"+hive:vdates AND +channel:(adam bethany dev notifications preet= share) AND at:[2014-07-10T21:28:07Z TO *]", "rows":"300"}}, "response":{"numFound":1,"start":0,"maxScore":1.0,"docs":[ { "hive":"vdates", "timestamp":1405027721000, "type":"MESSAGE", "id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1", "message_id":"2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1", "score":1.0}] }} 0 > incorrect 'missing required field' during update - document definitely ha= s it > -------------------------------------------------------------------------= ---- > > Key: SOLR-6251 > URL: https://issues.apache.org/jira/browse/SOLR-6251 > Project: Solr > Issue Type: Bug > Components: SolrCloud > Affects Versions: 4.8 > Environment: 4.8.0. Two nodes, SolrCloud, external ZK ensemble. A= ll on EC2. The two hosts are round-robin'd behind an ELB. > Reporter: Nathan Neulinger > Labels: replication > > Document added on solr1. We can see the distribute take place from solr1 = to solr2 and returning a success. Subsequent searches returning document, c= learly showing the field as being there. Later on, an update is done to add= to an element of the document - and the update fails. The update was sent = to solr2 instance.=20 > Schema marks the 'timestamp' field as required, so the initial insert sho= uld not work if the field isn't present. > Symptom is intermittent - we're seeing this randomly, with no warning or = triggering that we can see, but in all cases, it's getting the error in res= ponse to an update when the instance tries to distribute the change to the = other node.=20 > Searches that were run AFTER the update also show the field as being pres= ent in the document.=20 > Will add full trace of operations in the comments shortly. pcap captures = of ALL traffic for the two nodes on 8983 is available if requested.=20 -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org