lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shad Storhaug <s...@shadstorhaug.com>
Subject RE: Lucene.NET 4.8 is now in Beta - and we need your help!
Date Fri, 22 Jul 2016 02:21:59 GMT
Itamar,

> BoboBrowse seems to be discontinued, no?

The last commit was in May 2014. Not active, yes. But I am not sure about discontinued. It
is a bit confusing what is going on with the BoboBrowse project, though - the last few commits
had some major refactoring and bug fixes done, but there is no tag for them indicating that
they were ever released. Not to mention, I haven't found a way to contact the project owners
- I made a couple of attempts last year when I ported BoboBrowse 3.2.0, but they went unanswered.

I haven't tried to use the latest faceted search features of Lucene 4.8.0, but the main issue
in the past were lack of examples and seemingly a barebones implementation that didn't do
much on its own. BoboBrowse gets both the facet counts and document results in a single round
trip. It provides sorting, grouping, paging, a simple abstraction over the user's selections,
and has several more built-in facet types than Lucene. See http://stackoverflow.com/questions/1578890/is-there-a-faceted-search-framework-for-sql-server-net/28103881#28103881.
So, for those reasons (and because I have already done so for Lucene.Net 3.0.3) I am updating
the implementation to the corresponding version (4.0.2) so it works with the latest Lucene.Net.
I will probably also roll in changes from the latest commits to see if they are worthy of
release. If it turns out to progress no further or be able to compete with the built-in faceting
functionality of Lucene.Net, so be it.

It is sad how search engine functionality is sort of an ugly stepchild of the .NET world,
and I really don't understand why Microsoft would rather invest some meager resources in a
port of a Java framework rather than creating a .NET optimized implementation in the core
framework. .NET has been around for nearly 15 years - there really should be built-in search
by now, but Lucene.Net is a close second.

Thanks for looking into AddAttribute. The way it is now seems non-intuitive, and doesn't seem
to work the same way as in Java, but I have updated my code and confirmed that it works.

I have managed to get all unit tests green by adding temporary classes to mock QueryParser
and StandardAnalyzer. BoboBrowse depends heavily on StandardAnalyzer, but it turns out that
just using the SimpleAnalyzer seems to be enough to make the tests pass, since it is really
designed to work with any Analyzer implementation. QueryParser is used only in a few places,
and there is just one dependent test that is only checking for success and no failures, so
for the time being I have returned a MatchAllDocsQuery to get it to pass.

I have submitted an ICLA already and plan on starting small with some API inconsistencies,
but wouldn't mind helping out on StandardAnalyzer and QueryParser to move things along. It
seems like most of the code is already done on StandardAnalyzer and its dependencies, but
it isn't included in the Lucene.Net.Analysis.Common project file.

Just wondering what your take is on these:

public class IDisposableThreadLocal<T> : IDisposable // A class that is named like an
interface? Confusing for us C# guys.

And

public SerializableField(IndexableField field)
{
	name = field.Name(); // Method
	if (field.NumericValue != null)
	{
		fieldsData = field.NumericValue; // Property
	}
	else if (field.StringValue != null)
	{
		fieldsData = field.StringValue; // Property
	}
	else if (field.BinaryValue() != null)
	{
		fieldsData = field.BinaryValue().Bytes; // Method
	}
	else
	{
		throw new NotSupportedException("Doesn't support this field type so far");
	}
}

It seems like the above should all be properties (or all methods, but they should at least
be consistent).

One other thing - is there an issue tracker for Lucene.Net? I can't seem to find one.

Thanks,
NightOwl888

-----Original Message-----
From: itamar.synhershko@gmail.com [mailto:itamar.synhershko@gmail.com] On Behalf Of Itamar
Syn-Hershko
Sent: Friday, July 22, 2016 4:50 AM
To: user@lucenenet.apache.org
Subject: Re: Lucene.NET 4.8 is now in Beta - and we need your help!

Hi Shad,

Thanks for sharing this! BoboBrowse seems to be discontinued, no? worth pointing you at https://github.com/apache/lucenenet/tree/master/src/Lucene.Net.Facet
which is fully ported but with some failing tests - recent Lucene released support advanced
faceting directly from the core (via the Facet subproject) so no external libraries are really
needed for that. Maybe that's why BoboBrowse was never upgraded to 4.8.x.

With regards to the failure you describe, calling payloadAttr = base.AddAttribute<IPayloadAttribute>();
will work (note the use of the interface as the type parameter).

I don't remember why we did it this way, and will review it once again soon
- but this works just fine when used like this :) I think it was done this way because of
some inheritance related differences between Java and .NET, but I'll get back to you on this
soon.

The QueryParser and Analyzer classes are now in subprojects - that are still in the process
of being ported - and any help there will be greatly appreciated.

Cheers,

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko> Freelance Developer
& Consultant Lucene.NET committer and PMC member

On Wed, Jul 20, 2016 at 4:46 AM, Shad Storhaug <shad@shadstorhaug.com>
wrote:

> Itamar,
>
> Thanks for all of your hard work on this (not just to you, but to all 
> whom are involved).
>
> After this announcement, I began work on porting BoboBrowse.NET 4.0.2 
> (the latest official release) faceted search engine to see if it will 
> work with Lucene.Net 4.8.0. bobo-browse 4.0.2 was based on Lucene 
> 4.3.0, so I am hopeful it will work with 4.8.0.
>
> Anyway, long story short I have discovered a bug that existed in 
> Lucene.NET 3.0.3 has not been squashed. The test setup for most of the 
> BoboBrowse.NET unit tests depends on a TokenStream derived class, and 
> when building an index with this class it is failing.
>
> I have created a branch here to outline the problem and create a 
> working sample of the bug:
> https://github.com/NightOwl888/BoboBrowse.Net/blob/4.0.2-lucene-attr-i
> ssue/src/BoboBrowse.Net/BoboBrowse.Tests/BoboTestCase.cs#L128-L133
>
>
> If I try to run it with the original lines (130-133), I get an exception.
>
> System.ArgumentException : AddAttribute() only accepts an interface 
> that extends IAttribute, but 
> Lucene.Net.Analysis.Tokenattributes.PayloadAttribute does not fulfil 
> this contract.
> Result StackTrace:
> at Lucene.Net.Util.AttributeSource.AddAttribute[T]()
> at BoboBrowse.Net.BoboTestCase.MetaSizeTokenStream..ctor(Term term, 
> Int32
> size) in
> f:\Projects\BoboBrowse.Net\src\BoboBrowse.Net\BoboBrowse.Tests\BoboTes
> tCase.cs:line
> 133
> at BoboBrowse.Net.BoboTestCase.BuildMetaSizePayloadField(Term term, 
> Int32
> size) in
> f:\Projects\BoboBrowse.Net\src\BoboBrowse.Net\BoboBrowse.Tests\BoboTes
> tCase.cs:line
> 178
> at BoboBrowse.Net.BoboTestCase.BuildData() in 
> f:\Projects\BoboBrowse.Net\src\BoboBrowse.Net\BoboBrowse.Tests\BoboTes
> tCase.cs:line
> 195
> at BoboBrowse.Net.BoboTestCase.CreateIndex() in 
> f:\Projects\BoboBrowse.Net\src\BoboBrowse.Net\BoboBrowse.Tests\BoboTes
> tCase.cs:line
> 439
> at BoboBrowse.Net.BoboTestCase.Init() in 
> f:\Projects\BoboBrowse.Net\src\BoboBrowse.Net\BoboBrowse.Tests\BoboTes
> tCase.cs:line
> 86
>
> I checked the source and the PayloadAttribute definitely implements 
> IAttribute, so apparently the code that checks for the interface is broken.
>
> This was also broken in Lucene.Net 3.0.3 in a similar way. However, I 
> was able to work around the problem by calling AddAttributeImpl 
> instead of AddAttribute<T> (lines 137-142). So far this also seems to 
> be working in Lucene.Net 4.8.0.
>
> Anyway, I would appreciate if someone can take a look at this.
>
> In addition, I sure hope you don't plan on releasing without the 
> QueryParser and StandardAnalyzer classes!! These are referenced from 
> BoboBrowse, and for now I have built stubs to make the project compile.
>
>
> Also, if anyone is interested in an alternative faceted search 
> implementation than the built-in one, I could use assistance:
>
> 1. Getting unit tests to pass
> 2. Making the API consistent with the Lucene.Net API (properties vs
> methods)
> 3. Cleaning up the Close vs Dispose issues, also which should be 
> similar to Lucene.Net 4. Reporting bugs once it is stable enough to 
> identify them
>
> Now accepting pull requests on the 4.0.2 branch.
> https://github.com/NightOwl888/BoboBrowse.Net/tree/4.0.2 Do note that 
> the ReadMe has not yet been updated to reflect the new version.
>
> Thanks,
> NightOwl888
>
>
> -----Original Message-----
> From: itamar.synhershko@gmail.com [mailto:itamar.synhershko@gmail.com] 
> On Behalf Of Itamar Syn-Hershko
> Sent: Thursday, July 7, 2016 11:08 PM
> Subject: Lucene.NET 4.8 is now in Beta - and we need your help!
>
> Hi folks,
>
> I just posted this -
>
> http://code972.com/blog/2016/07/98-lucene-net-4-8-is-in-beta-and-we-ne
> ed-your-help
> - with the latest details and instructions on the upcoming Lucene.NET 
> release.
>
> We would love to see more involvement from anyone interested in the 
> project.
>
> Cheers,
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko> 
> Freelance Developer & Consultant Lucene.NET committer and PMC member
>
Mime
View raw message