Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6C3AA1124A for ; Wed, 21 May 2014 18:37:42 +0000 (UTC) Received: (qmail 18334 invoked by uid 500); 21 May 2014 18:37:41 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 18270 invoked by uid 500); 21 May 2014 18:37:41 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 18263 invoked by uid 99); 21 May 2014 18:37:41 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 May 2014 18:37:41 +0000 Date: Wed, 21 May 2014 18:37:41 +0000 (UTC) From: "Hoss Man (JIRA)" To: dev@lucene.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (SOLR-5285) Solr response format should support child Docs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SOLR-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-5285: --------------------------- Attachment: SOLR-5285.patch Hey Varun, I didn't get very far digging into your patch, because i started by looking at your new randomized test in SolrExampleTests and encountered some problems... 1) the first time i tried running your new randomized test, i got an NPE -- it didn't reproduce reliable though, because your test called "new Random()" instead of leveraging the test-framework ("ant precommit" will warn you about stuff like this) 2) Side note: there's no need to randomize which response parser is used when you add test methods to "SolrExampleTests" -- every method there gets picked up automatically by the subclasses which ensure they are all run with every writer/parser. 3) When started looking into fixing the use of random() in your test, I realized that the assertions in the test weren't very strong. What i was refering to in my earlier comment was having a test that attempted to use the transformer on a result set that included docs with children, and docs w/o children; and asserting that every child returned really was a decendent of the specified doc by comparing with what we _know_ for a fact we indexed -- your test wasn't really doing any of that. In the attached patch, i've overhauled {{SolrExampleTests.testChildDoctransformer()}} along the lines of what i was describing, but this has exposed a ClassCastException in the transformer. I haven't had a chance to dig into what's happening, but for some odd reason it only seems to manifest itself when the XML Response Writer is used... {noformat} hossman@frisbee:~/lucene/dev/solr/solrj$ ant test -Dtests.method=testChildDoctransformer -Dtests.seed=720251997BEC4F70 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=Pacific/Fiji -Dtests.file.encoding=UTF-8 ... [junit4] 2> 11768 T20 C1 oasc.SolrException.log ERROR null:java.lang.ClassCastException: org.apache.lucene.document.Field cannot be cast to java.lang.String [junit4] 2> at org.apache.solr.response.transform.ChildDocTransformer.transform(ChildDocTransformerFactory.java:142) [junit4] 2> at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:254) [junit4] 2> at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:172) [junit4] 2> at org.apache.solr.response.XMLWriter.writeResponse(XMLWriter.java:111) [junit4] 2> at org.apache.solr.response.XMLResponseWriter.write(XMLResponseWriter.java:40) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:760) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) [junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:368) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [junit4] 2> at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628) [junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:744) [junit4] 2> [junit4] 2> 11774 T12 oas.SolrTestCaseJ4.tearDown ###Ending testChildDoctransformer [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SolrExampleStreamingTest -Dtests.method=testChildDoctransformer -Dtests.seed=720251997BEC4F70 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=Pacific/Fiji -Dtests.file.encoding=UTF-8 [junit4] ERROR 5.76s J2 | SolrExampleStreamingTest.testChildDoctransformer <<< ... [junit4] 2> 16262 T39 C1 oasc.SolrException.log ERROR null:java.lang.ClassCastException: org.apache.lucene.document.Field cannot be cast to java.lang.String [junit4] 2> at org.apache.solr.response.transform.ChildDocTransformer.transform(ChildDocTransformerFactory.java:142) [junit4] 2> at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:254) [junit4] 2> at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:172) [junit4] 2> at org.apache.solr.response.XMLWriter.writeResponse(XMLWriter.java:111) [junit4] 2> at org.apache.solr.response.XMLResponseWriter.write(XMLResponseWriter.java:40) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:760) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) [junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:368) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [junit4] 2> at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628) [junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:744) [junit4] 2> [junit4] 2> 16264 T30 oas.SolrTestCaseJ4.tearDown ###Ending testChildDoctransformer [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SolrExampleXMLTest -Dtests.method=testChildDoctransformer -Dtests.seed=720251997BEC4F70 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=Pacific/Fiji -Dtests.file.encoding=UTF-8 [junit4] ERROR 0.96s J1 | SolrExampleXMLTest.testChildDoctransformer <<< ... [junit4] Tests with failures: [junit4] - org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest.testChildDoctransformer [junit4] - org.apache.solr.client.solrj.SolrExampleXMLTest.testChildDoctransformer {noformat} > Solr response format should support child Docs > ---------------------------------------------- > > Key: SOLR-5285 > URL: https://issues.apache.org/jira/browse/SOLR-5285 > Project: Solr > Issue Type: New Feature > Reporter: Varun Thacker > Fix For: 4.9, 5.0 > > Attachments: SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, SOLR-5285.patch, javabin_backcompat_child_docs.bin > > > Solr has added support for taking childDocs as input ( only XML till now ). It's currently used for BlockJoinQuery. > I feel that if a user indexes a document with child docs, even if he isn't using the BJQ features and is just searching which results in a hit on the parentDoc, it's childDocs should be returned in the response format. > [~hossman_lucene@fucit.org] on IRC suggested that the DocTransformers would be the place to add childDocs to the response. > Now given a docId one needs to find out all the childDoc id's. A couple of approaches which I could think of are > 1. Maintain the relation between a parentDoc and it's childDocs during indexing time in maybe a separate index? > 2. Somehow emulate what happens in ToParentBlockJoinQuery.nextDoc() - Given a parentDoc it finds out all the childDocs but this requires a childScorer. > Am I missing something obvious on how to find the relation between a parentDoc and it's childDocs because none of the above solutions for this look right. -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org