incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: [VOTE] Merge lucene-4.0.0 branch to master
Date Tue, 23 Oct 2012 21:37:11 GMT
Hmm, I will take a look at that one next.

Aaron

On Tue, Oct 23, 2012 at 5:20 PM, Patrick Hunt <phunt@apache.org> wrote:
> Thanks Aaron. The other failing test "BlurClusterTest" is somehow due
> to the directory used. "./tmp/cluster". If I change to
> "file://tmp/cluster" the test passes. Any ideas? Seems somehow related
> to using relative paths?
>
> Patrick
>
> On Tue, Oct 23, 2012 at 2:13 PM, Aaron McCurry <amccurry@gmail.com> wrote:
>> Found it, the test did not setup the indexing options correctly.  I
>> have committed a fix for the test.
>>
>> Aaron
>>
>> On Tue, Oct 23, 2012 at 5:08 PM, Aaron McCurry <amccurry@gmail.com> wrote:
>>> After cleaning up the test, I have gotten the same NPE.  Strange
>>> behavior, still working on why.
>>>
>>> Aaron
>>>
>>> On Tue, Oct 23, 2012 at 3:06 PM, Patrick Hunt <phunt@apache.org> wrote:
>>>> NP. here's the output. I'm on ubuntu 12.04. 1.6.0_26
>>>>
>>>> "mvn clean test" results in: (I also removed the tmp directories
>>>> manually, btw, we should move this to mvn target  dir)
>>>>
>>>> -------------------------------------------------------------------------------
>>>> Test set: org.apache.blur.utils.TermDocIterableTest
>>>> -------------------------------------------------------------------------------
>>>> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.005
>>>> sec <<< FAILURE!
>>>> testTermDocIterable(org.apache.blur.utils.TermDocIterableTest)  Time
>>>> elapsed: 0.005 sec  <<< ERROR!
>>>> java.lang.NullPointerException
>>>>         at org.apache.blur.utils.TermDocIterable.getNext(TermDocIterable.java:82)
>>>>         at org.apache.blur.utils.TermDocIterable.access$000(TermDocIterable.java:29)
>>>>         at org.apache.blur.utils.TermDocIterable$1.<init>(TermDocIterable.java:48)
>>>>         at org.apache.blur.utils.TermDocIterable.iterator(TermDocIterable.java:47)
>>>>         at org.apache.blur.utils.TermDocIterableTest.testTermDocIterable(TermDocIterableTest.java:65)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>>         at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>>>         at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>>>         at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>>>         at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>>>         at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>>>>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
>>>>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>>>>         at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>>>>         at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>>>>         at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>>>>         at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>>>>         at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>>>>         at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>>>>         at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
>>>>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
>>>>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>>         at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
>>>>         at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
>>>>         at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
>>>>         at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
>>>>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
>>>>
>>>>
>>>> On Tue, Oct 23, 2012 at 12:02 PM, Aaron McCurry <amccurry@gmail.com>
wrote:
>>>>> Sorry, just missed that message.  Hmm, I will look around and try to
>>>>> see if I can find something.  Thanks.
>>>>>
>>>>> Aaron
>>>>>
>>>>> On Tue, Oct 23, 2012 at 2:59 PM, Patrick Hunt <phunt@apache.org>
wrote:
>>>>>> this is null in termdocsitertest
>>>>>>
>>>>>>         DocsEnum termDocs = atomicReader.termDocsEnum(new Term("id",
>>>>>> Integer.toString(id)));
>>>>>>
>>>>>> due to fields() being null in termDocsEnum method
>>>>>>
>>>>>> I don't see why yet though. Given the segment file exists on the
>>>>>> filesystem, etc...
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> On Tue, Oct 23, 2012 at 11:50 AM, Aaron McCurry <amccurry@gmail.com>
wrote:
>>>>>>> Trying to reproduce on Ubuntu.
>>>>>>>
>>>>>>> On Tue, Oct 23, 2012 at 1:58 PM, Patrick Hunt <phunt@apache.org>
wrote:
>>>>>>>> Hm, I just updated and I'm seeing two errors (which is 1
less issue
>>>>>>>> than before):
>>>>>>>>
>>>>>>>>   testTermDocIterable(org.apache.blur.utils.TermDocIterableTest)
>>>>>>>>   org.apache.blur.thrift.BlurClusterTest: java.lang.NullPointerException
>>>>>>>>
>>>>>>>> Let me look and see if I can at least determine what the
underlying
>>>>>>>> problems are.
>>>>>>>>
>>>>>>>> Patrick
>>>>>>>>
>>>>>>>> On Tue, Oct 23, 2012 at 10:12 AM, Aaron McCurry <amccurry@gmail.com>
wrote:
>>>>>>>>> I ran into some errors with ZookeeperClusterStatusTest
tests and have
>>>>>>>>> resolved the issues I found.  All units tests pass on
OSX, I have not
>>>>>>>>> had a chance to run them on Linux yet.  I also fixed
the nasty NPE
>>>>>>>>> exception on the BlurClusterTest (it was affecting the
functional
>>>>>>>>> tests as well).  I ran a few burn-in tests on a VM running
a 2
>>>>>>>>> controller + 3 shard server Blur cluster.  The tests
included loaded
>>>>>>>>> data as fast as possibly while running searches against
that data as
>>>>>>>>> fast as possible.  The tests ran without issue (basically
like they
>>>>>>>>> did before the upgrade to Lucene 4).  I feel like the
code is in a
>>>>>>>>> good state at this point.  I'm going to merge this code
to master and
>>>>>>>>> create another branch to begin modifying the RPC API.
>>>>>>>>>
>>>>>>>>> Anyone have any objections?
>>>>>>>>>
>>>>>>>>> Aaron
>>>>>>>>>
>>>>>>>>> On Mon, Oct 22, 2012 at 8:29 PM, Patrick Hunt <phunt@apache.org>
wrote:
>>>>>>>>>> On Mon, Oct 22, 2012 at 5:23 PM, Aaron McCurry <amccurry@gmail.com>
wrote:
>>>>>>>>>>> Hmm.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 22, 2012 at 8:17 PM, Patrick Hunt
<phunt@apache.org> wrote:
>>>>>>>>>>>> Sounds good to me.
>>>>>>>>>>>>
>>>>>>>>>>>> Not sure if anyone else is seeing this but
the unit tests are not
>>>>>>>>>>>> passing for me on ubuntu. I see one failure
and two errors.
>>>>>>>>>>>>
>>>>>>>>>>>> Failed tests:
>>>>>>>>>>>>    testSafeModeSetInFuture(org.apache.blur.manager.clusterstatus.ZookeeperClusterStatusTest)
>>>>>>>>>>>
>>>>>>>>>>> Haven't seen this.
>>>>>>>>>>>
>>>>>>>>>>>> Tests in error:
>>>>>>>>>>>>   testTermDocIterable(org.apache.blur.utils.TermDocIterableTest)
>>>>>>>>>>>
>>>>>>>>>>> This either.
>>>>>>>>>>>
>>>>>>>>>>>>   org.apache.blur.thrift.BlurClusterTest:
java.lang.NullPointerException
>>>>>>>>>>>
>>>>>>>>>>> I think I have been seeing this one during some
functional tests.
>>>>>>>>>>> Haven't figured out the cause yet, but it seems
like it's a nasty
>>>>>>>>>>> threading problem.  Because when I drop the mutate
threads back 1
>>>>>>>>>>> everything works fine.  However the test was
passing on OSX.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Just me or is this expected?
>>>>>>>>>>>
>>>>>>>>>>> Not expected.  I'm going to setup a VM on computer
to run tests in
>>>>>>>>>>> Linux as well.
>>>>>>>>>>
>>>>>>>>>> Ok. Let me know how it goes and I can try and debug
it a bit, although
>>>>>>>>>> you're running much faster than I can at this point.
;-) Definitely
>>>>>>>>>> let me know if you can't reproduce it and I'll dig
into it for sure.
>>>>>>>>>>
>>>>>>>>>> Patrick
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Patrick
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Oct 21, 2012 at 10:38 AM, Aaron McCurry
<amccurry@gmail.com> wrote:
>>>>>>>>>>>>> We can fix the jira issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Oct 21, 2012 at 1:36 PM, Garrett
Barton
>>>>>>>>>>>>> <garrett.barton@gmail.com> wrote:
>>>>>>>>>>>>>> Sounds good to me Aaron, call it
0.2. Does that mess up Jira if you have
>>>>>>>>>>>>>> things scheduled against releases?
>>>>>>>>>>>>>> On Oct 21, 2012 9:44 AM, "Aaron McCurry"
<amccurry@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ok, I think it will be some time
before all the changes for the new
>>>>>>>>>>>>>>> api are in place and fully functional.
 So perhaps we should merge the
>>>>>>>>>>>>>>> lucene-4.0.0 branch into master
and fix whatever bugs are found.  I
>>>>>>>>>>>>>>> did some system testing yesterday
and only found one big issue.  There
>>>>>>>>>>>>>>> seems to be a threading problem
with the BlurAnalyzer.  If a single
>>>>>>>>>>>>>>> instance is in use across multiple
threads some weird behaviors
>>>>>>>>>>>>>>> happen.  Otherwise everything
else seems to work, normally (I will
>>>>>>>>>>>>>>> create a jira issue).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If we do merge the lucene-4.0.0
branch, I feel like we should change
>>>>>>>>>>>>>>> the version to 0.2.  The reason
is, the indexes in 0.1.x are not going
>>>>>>>>>>>>>>> to be backwards compatible (at
least not with out some work).  Does
>>>>>>>>>>>>>>> anyone have any strong feelings
on this?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Aaron
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sat, Oct 20, 2012 at 10:10
PM, Gagan Juneja
>>>>>>>>>>>>>>> <gagandeepjuneja@gmail.com>
wrote:
>>>>>>>>>>>>>>> > I agree with Garrett. We
can merge this branch to the place from where we
>>>>>>>>>>>>>>> > cut it. Again as Garrett
said If we want to keep only new api thing then
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> > can merge it to master as
well.
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > Regards,
>>>>>>>>>>>>>>> > Gagan
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > On Sat, Oct 20, 2012 at
9:50 PM, Garrett Barton <
>>>>>>>>>>>>>>> garrett.barton@gmail.com>wrote:
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> >> I guess it depends on
if your planning a 1.4 release with lucene 4. If
>>>>>>>>>>>>>>> yes
>>>>>>>>>>>>>>> >> then merge and work
towards making everything functional. If not then
>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>> >> the 1.3.x in master
for bug fixing or whatnot and merge this branch into
>>>>>>>>>>>>>>> >> the new api one.
>>>>>>>>>>>>>>> >> On Oct 20, 2012 11:03
AM, "Aaron McCurry" <amccurry@gmail.com> wrote:
>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>> >> > I think that we
can merge the lucene-4.0.0 branch back into the
>>>>>>>>>>>>>>> >> > master, since tests
and code are compiling.  I haven't done any
>>>>>>>>>>>>>>> >> > functional testing
yet, but if much of the RPC and internals are going
>>>>>>>>>>>>>>> >> > to change I think
that it may be a waste of time to test and fix
>>>>>>>>>>>>>>> >> > everything that
we are about to change.  What do others think?
>>>>>>>>>>>>>>> >> >
>>>>>>>>>>>>>>> >> > Aaron
>>>>>>>>>>>>>>> >> >
>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>

Mime
View raw message