lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ishan Chattopadhyaya <ichattopadhy...@gmail.com>
Subject Re: Release 6.6
Date Tue, 16 May 2017 01:54:55 GMT
I attempted to build an RC, but failed repeatedly before realizing it was
due to, apart from test failures, LUCENE-7830 / LUCENE-6438. I've manually
cleared up the dead symlinks now. I'll build the RC on Tuesday, as I'm
about to wrap up the day's work.

On Mon, May 15, 2017 at 10:12 PM, Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:

> I wish to backport a fix from SOLR-8440 (last commit) to the release
> branch. It affects only the feature introduced in SOLR-8440. Please let me
> know if someone has any objections.
>
> Also, I'm planning to build the RC in another 3-4 hours. Please let me
> know if there's something that someone is working on which needs to get in
> before that.
>
> Thanks and regards,
> Ishan
>
>
> On Sun, May 14, 2017 at 5:02 PM, Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
>> Sure Steve! Thanks!
>>
>> On Sun, May 14, 2017 at 2:34 PM, Steve Rowe <sarowe@gmail.com> wrote:
>>
>>> Ishan,
>>>
>>> Okay to include https://issues.apache.org/jira/browse/LUCENE-7821 for
>>> 6.6?
>>>
>>> --
>>> Steve
>>> www.lucidworks.com
>>>
>>> > On May 12, 2017, at 12:51 PM, jim ferenczi <jim.ferenczi@gmail.com>
>>> wrote:
>>> >
>>> > Ok thanks Ishan.
>>> >
>>> > Le 12 mai 2017 18:44, "Ishan Chattopadhyaya" <
>>> ichattopadhyaya@gmail.com> a écrit :
>>> > Sure, Jim. Please go ahead!
>>> >
>>> > On Fri, May 12, 2017 at 10:01 PM, jim ferenczi <jim.ferenczi@gmail.com>
>>> wrote:
>>> > Hi,
>>> > Would be great to have https://issues.apache.org/jira
>>> /browse/LUCENE-7824 included for 6.6. Can I merge the fix this week end
>>> Ishan ?
>>> >
>>> > 2017-05-11 16:45 GMT+02:00 Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com>:
>>> > Done, Adrien. Thanks!
>>> >
>>> > On Thu, May 11, 2017 at 7:34 PM, Adrien Grand <jpountz@gmail.com>
>>> wrote:
>>> > Ishan, wdyt about running addVersion on the branch_6x/master as well?
>>> I think it would help realize that the 6.6 branch has been cut when looking
>>> at the CHANGES.txt file and not forget about backporting?
>>> >
>>> > Le mar. 9 mai 2017 à 17:58, Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com> a écrit :
>>> > I've cut the branch for 6.6. (branch_6_6). Please feel free to add
>>> bugfixes to the branch, if needed.
>>> > Planning to build the first RC on 15 May. Please let me know if there
>>> are any objections.
>>> >
>>> > Thanks,
>>> > Ishan
>>> >
>>> > On Tue, May 9, 2017 at 11:10 AM, Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com> wrote:
>>> > Planning to cut the release branch in another 10-12 hours.
>>> >
>>> > On Mon, May 1, 2017 at 4:00 PM, Martin Gainty <mgainty@hotmail.com>
>>> wrote:
>>> > i was wondering if there was a specific test for SpanPayloadCheckQuery
>>> method
>>> >
>>> > matches = payloadToMatch.get(upto).bytesEquals(payload);
>>> >
>>> >
>>> >
>>> > the only implementation i could locate was in collectLeaf of
>>> SpanPayloadCheckQuery
>>> >
>>> >
>>> > I will submit JIRA with Patch
>>> >
>>> >
>>> > as a CS *dinosaur* I am more familiar with LISP for dissecting
>>> sentence fragments (what we called phenomes) than current SEO
>>> implementations
>>> >
>>> >
>>> > Can you suggest a book to read to better understand Lucenes pattern
>>> dissection and match algorithms?
>>> >
>>> >
>>> > Many Thanks for helpful guidance
>>> > Martin
>>> > ______________________________________________
>>> >
>>> >
>>> >
>>> > From: Erik Hatcher <erik.hatcher@gmail.com>
>>> > Sent: Sunday, April 30, 2017 2:06 PM
>>> >
>>> > To: dev@lucene.apache.org
>>> > Subject: Re: Release 6.6
>>> >
>>> > Martin -
>>> >
>>> > I have to admit to still being unsure what the gist is here - is there
>>> a bug?   Apologies for not catching what you’re saying/showing here.
>>> Again, as far as I can tell SpanPayloadCheckQuery is working as expected
>>> now.
>>> >
>>> > I’m going to focus purely on SOLR-1485 this week to get it committed
>>> for 6.6.  If there is an issue to address with your work would you please
>>> open a JIRA and include your patch there?
>>> >
>>> > Thanks,
>>> > Erik
>>> >
>>> >
>>> >> On Apr 30, 2017, at 7:47 AM, Martin Gainty <mgainty@hotmail.com>
>>> wrote:
>>> >>
>>> >> Mornin' Erik
>>> >>
>>> >> there is a collectLeaf  override in org.apache.lucene.queries.payloads.TestPayloadSpans
>>> ..but its never called:
>>> >>
>>> >> static class VerifyingCollector implements SpanCollector {
>>> >>     List<BytesRef> payloads = new ArrayList<>();
>>> >>     @Override
>>> >>     public void collectLeaf(PostingsEnum postings, int position, Term
>>> term) throws IOException {
>>> >>      ....
>>> >>      }
>>> >> }
>>> >>
>>> >> the modification in org.apache.lucene.queries.payloads.TestPayloadCheckQuery
>>> tests collectLeaf for query
>>> >>
>>> >> //initialise term
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231
>>> before term1=new org.apache.lucene.index.Term('field','withPayload')");
>>> >> org.apache.lucene.index.Term term1=new org.apache.lucene.index.Term("field",
>>> "withPayload");
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233
>>> term1="+term1);
>>> >>
>>> >> //assume position is 5
>>> >>     int position=5;
>>> >>     log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 235
>>> position="+position);
>>> >>
>>> >>     BytesRef pay = new BytesRef("pos: " + position);
>>> >>     log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 237
>>> pay="+pay);
>>> >>
>>> >> //build spanQuery with term parameter
>>> >>     org.apache.lucene.search.spans.SpanQuery spanQuery1 = new
>>> SpanTermQuery(term1);
>>> >>     log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 239
>>> spanQuery1="+spanQuery1);
>>> >>
>>> >> //add BytesRef to payloadToMatch list
>>> >>     java.util.List<org.apache.lucene.util.BytesRef>
>>> payloadToMatch=new java.util.ArrayList<org.apache
>>> .lucene.util.BytesRef>();
>>> >>     log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 241
>>> payloadToMatch="+payloadToMatch);
>>> >>     payloadToMatch.add(pay);
>>> >>
>>> >> //build SpanPayloadCheckQuery
>>> >> query=new org.apache.lucene.queries.payloads.SpanPayloadCheckQuery(
>>> >> (org.apache.lucene.search.spans.SpanQuery)query,
>>> >> (java.util.List<org.apache.lucene.util.BytesRef>)payloadToMatch);
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 249
>>> query="+query);
>>> >>
>>> >> //build lucene Directory (TODO: we should use an existing directory
>>> with REAL test-data)
>>> >> org.apache.lucene.store.Directory ram =
>>> newDirectory(com.carrotsearch.randomizedtesting.RandomizedCo
>>> ntext.current().getRandom());
>>> >>
>>> >> //build SegmentReader from Directory
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 251
>>> ram="+ram);
>>> >> SegmentReader reader = getOnlySegmentReader(org.apach
>>> e.lucene.index.DirectoryReader.open(ram));
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 253
>>> reader="+reader);
>>> >>
>>> >> //populate SlowCompositeReaderWrapper with SegmentReader
>>> >>     org.apache.lucene.index.LeafReader sr =
>>> org.apache.lucene.index.SlowCompositeReaderWrapper.wrap(reader);
>>> >>     log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 255
>>> sr="+sr);
>>> >>
>>> >> //add term to SegmentReader postings
>>> >> org.apache.lucene.index.PostingsEnum postings = sr.postings(term1,
>>> PostingsEnum.PAYLOADS);
>>> >>
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257
>>> before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where
>>> postings="+postings);
>>> >>
>>> >> //display the parameters for collectLeaf method for query:
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258
>>> before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where
>>> position="+position);
>>> >>
>>> >> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259
>>> before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where term1="+term1);
>>> >>     try
>>> >>     { //public void collectLeaf(org.apache.lucene.index.PostingsEnum
>>> postings, int position,       //org.apache.lucene.index.Term term)
>>> throws java.io.IOException {
>>> >> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1);
>>> >> }
>>> >> catch(java.io.IOException ioe) { log.debug("TestPayloadCheckQuery::testSpanPayloadCheck
>>> LINE 264 query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws
>>> IOException ="+ioe.getMessage()); }
>>> >>
>>> >> collectLeaf is the only method that compares
>>> matches=payloadToMatch.get(upto).bytesEquals(payload)
>>> >> to determine "match"
>>> >>
>>> >> unless of course match between individual payload and payloadToMatch
>>> is tested elsewhere ?
>>> >>
>>> >> WDYT?
>>> >> Martin
>>> >> ______________________________________________
>>> >>
>>> >>
>>> >>
>>> >> From: Erik Hatcher <erik.hatcher@gmail.com>
>>> >> Sent: Saturday, April 29, 2017 7:58 PM
>>> >> To: Lucene/Solr dev
>>> >> Subject: Re: Release 6.6
>>> >>
>>> >> Martin -
>>> >>
>>> >> Interesting test but I couldn’t copy/paste it in to try it out as
it
>>> uses some logging and APIs that aren’t already in the project and classpath
>>> of these lucene tests within that queries module (within IntelliJ, mind
>>> you).
>>> >>
>>> >> I was able to resolve the misunderstanding (and .equals/.hashCode
>>> bugs) so everything is working as expected with SpanPayloadCheckQuery for
>>> me now.   Do your tests point out an issue?   Or confirming that it is
>>> working properly at a lower level?
>>> >>
>>> >> Erik
>>> >>
>>> >>
>>> >>> On Apr 29, 2017, at 9:08 AM, Martin Gainty <mgainty@hotmail.com>
>>> wrote:
>>> >>>
>>> >>> Martin Gainty has shared a OneDrive file with you. To view it,
click
>>> the link below.
>>> >>>
>>> >>> TestPayloadCheckQuery.java
>>> >>> attached
>>> >>>
>>> >>> I coded this last nite so it is "quick and dirty"
>>> >>>
>>> >>> in any case let me know if  testSpanPayloadCheck() method
>>> modification properly addresses your situation
>>> >>>
>>> >>> Thanks!
>>> >>> Martin
>>> >>> ______________________________________________
>>> >>>
>>> >>>
>>> >>>
>>> >>> From: Erik Hatcher <erik.hatcher@gmail.com>
>>> >>> Sent: Saturday, April 29, 2017 8:58 AM
>>> >>> To: dev@lucene.apache.org
>>> >>> Subject: Re: Release 6.6
>>> >>>
>>> >>> Martin - there is a test class specifically for this query.   See
>>> the recent commits I've made to test it further fixing .equals/.hashCode
>>> and rewrite.
>>> >>>
>>> >>> Can you send your full test code?
>>> >>>
>>> >>>    Erik
>>> >>>
>>> >>> On Apr 29, 2017, at 07:32, Martin Gainty <mgainty@hotmail.com>
>>> wrote:
>>> >>>
>>> >>>> when Erik mentioned he couldnt get SpanPayloadCheckQuery to
work I
>>> was about to reply just run that TestCase
>>> >>>> (until i discovered there wasnt any!)
>>> >>>>
>>> >>>> i'll start the bidding at 1 dinar for TestCase
>>> org.apache.lucene.queries.payloads.TestPayloadCheckQuery mod:
>>> >>>>   @org.junit.Test
>>> >>>>   public void testSpanPayloadCheck() throws Exception
>>> >>>>
>>> >>>>     //now lets test the collectLeaf for query
>>> >>>>     //of course calling Base Class SpanPayloadCheckQuery to
>>> construct query
>>> >>>>
>>> >>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE
257
>>> before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where
>>> postings="+postings);
>>> >>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE
258
>>> before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where
>>> position="+position);
>>> >>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE
259
>>> before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) where term1="+term1);
>>> >>>>
>>> >>>>     try
>>> >>>>     { //test public void collectLeaf(org.apache.lucene.index.PostingsEnum
>>> postings, int position,              //org.apache.lucene.index.Term term)
>>> throws java.io.IOException {
>>> >>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1);
>>> >>>> }
>>> >>>> catch(java.io.IOException ioe) { log.error("TestPayloadCheckQuery::testSpanPayloadCheck
>>> LINE 264 query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings,
>>> (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws
>>> IOException ="+ioe.getMessage()); }
>>> >>>>
>>> >>>> unless of course anyone has a better idea to test collectLeaf
?
>>> >>>> Martin
>>> >>>> ______________________________________________
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> From: Erik Hatcher <erik.hatcher@gmail.com>
>>> >>>> Sent: Tuesday, April 25, 2017 7:57 PM
>>> >>>> To: dev@lucene.apache.org
>>> >>>> Subject: Re: Release 6.6
>>> >>>>
>>> >>>> Probably no bribe needed, but an updated patch would be a good
>>> start (or will the 2.5 year old patch still apply and be acceptable as-is?)
>>> >>>>
>>> >>>> Erik
>>> >>>>
>>> >>>>> On Apr 25, 2017, at 7:52 PM, Walter Underwood <
>>> wunder@wunderwood.org> wrote:
>>> >>>>>
>>> >>>>> Who do I have to bribe to get SOLR-629 included?
>>> >>>>>
>>> >>>>> https://issues.apache.org/jira/browse/SOLR-629
>>> >>>>>
>>> >>>>> wunder
>>> >>>>> Walter Underwood
>>> >>>>> wunder@wunderwood.org
>>> >>>>> http://observer.wunderwood.org/  (my blog)
>>> >>>>>
>>> >>>>>
>>> >>>>>> On Apr 25, 2017, at 10:46 AM, Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com> wrote:
>>> >>>>>>
>>> >>>>>> Hi,
>>> >>>>>> We have lots of changes, optimizations, bug fixes for
6.6. I'd
>>> like to propose a 6.6 release (perhaps the last 6x non-bug-fix release
>>> before 7.0 release).
>>> >>>>>>
>>> >>>>>> I can volunteer to release this, and I can cut the release
branch
>>> around 4th May, in order to let some time for devs to finish current issues.
>>> >>>>>>
>>> >>>>>> What do you think?
>>> >>>>>>
>>> >>>>>> Regards,
>>> >>>>>> Ishan
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>

Mime
View raw message