lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shad Storhaug <>
Subject RE: API Work/Stabilization Update
Date Sat, 25 Mar 2017 14:04:59 GMT

> We will probably want to build out .nuspec files to get all the nuget stuff right for
these projects -- I don't think the generation will work for us to get things quite right.

Connie has set us up to use .json files instead of .nuspec files to generate the NuGet packages
(the new way instead of the old way). The build script Build.ps1 does it all (it even has
help documentation), but it is missing an option to override versioning. Ideally we would
be able to override the version that is in the .json file with an environment variable (which
you can pass from TeamCity), and be able to override that on the CLI for local "one-off" builds.
 See the build instructions on #191:

> Regarding the nuget key -- that plan works for me, the trick is I don't have the key
to add to myget.

I don't know what order the infrastructure was setup in on your end, but my thought was that
if someone had previously pushed from MyGet to NuGet the key is probably already configured
there. But yea, you would need access to MyGet to confirm.

> I would love to start beating on that a bit but the .net core version seems to want NUnit
3.5+ which needs to be added to the project to run.

I will take a look at your pull request, but I think this is a symptom of trying to run using
the older tooling. The Build.ps1 script already has the ability to test, and all of the tooling
is there to do it (I think - maybe I should do a fresh clone to be sure). It does have some
prerequisites, though (see #191). It builds both the .NET Framework and .NET Core versions
and packages them into NuGet.

Per #191: Hopefully Lucene.Net.sln can be removed in the future because the .NET Core projects
compile for .NET 4.5.1 already.

So I think the aim is to eventually eliminate those .csproj files (and for that matter .nuspec
files) and use strictly .json files for project configuration going forward.

Shad Storhaug (NightOwl888)

-----Original Message-----
From: Wyatt Barnett [] 
Sent: Saturday, March 25, 2017 5:00 AM
Subject: Re: API Work/Stabilization Update

Shad -- the overall plan sounds good. We will probably want to build out .nuspec files to
get all the nuget stuff right for these projects -- I don't think the generation will work
for us to get things quite right.

Regarding the nuget key -- that plan works for me, the trick is I don't have the key to add
to myget. Come to think of it I don't think I have the proverbial keys to the myget page either
but I think Martin can help us out there.

Buffers could be the issue on the tests -- I've long suspected that or I/O causing the meltdown,
I just haven't been able to reproduce. I would love to start beating on that a bit but the
.net core version seems to want NUnit 3.5+ which needs to be added to the project to run.
If you get that added I can start beating on the test problems a bit more.

Thanks for all your hard work putting this together, let me know how I can help you get it
out the proverbial door.

On Fri, Mar 24, 2017 at 9:34 AM Shad Storhaug <> wrote:

> Wyatt,
> Thanks. Actually, I was thinking this should go in a few steps instead 
> of
> one:
> 1. Merge #203.
> 2. Change the pre-release label to "beta2" and work out any issues to 
> build/push to MyGet (might take a few tries) 3. Update the README and 
> CONTRIBUTING pages 4. Push the package to NuGet
> I have always just used the control panel at MyGet to push upstream to 
> NuGet, and it is capable of storing someone's key so the person who 
> pushes it doesn't actually need it.
> As far as the tests burning down are concerned, I discovered that some 
> of them write so much "verbose" data that they overflow NUnit's buffer 
> and cause it to crash (sometimes this even causes Visual Studio to 
> crash locally). I think I have found all of the tests in the Core that 
> were causing this and hard-coded them to set verbose off (with the 
> ability to manually override), but I noticed that there are still 
> tests in Analysis.Common that can cause it to crash. I haven't 
> investigated if there is a setting in NUnit to increase the buffer 
> size, which might be a better fix, but I could probably track down the 
> rest of the tests that are causing this before the merge.
> Thanks,
> Shad Storhaug (NightOwl888)
> -----Original Message-----
> From: Wyatt Barnett []
> Sent: Friday, March 24, 2017 8:14 PM
> To:
> Subject: Re: API Work/Stabilization Update
> I'm about and happy to update when you are ready.
> I think the build should start working for 203 if you add the 
> nunit-console nuget package. At least work in the sense of the build 
> will start. I'm still chasing those build time bugbears I haven't been 
> able to slay yet.
> As for getting to -- who has the key? I've never had access 
> to it so I'm not sure we can update what is out there.
> On Fri, Mar 24, 2017 at 09:10 Shad Storhaug <> wrote:
> > Wyatt, Prescott, Itamar, anybody? We've almost got all of the tests 
> > passing on #203 and would like to merge to master and release it 
> > with the new pre-release label "beta2".
> >
> > If there is nobody available to get the build running after merging, 
> > could someone give me access to TeamCity to work on it?
> >
> > Thanks,
> > Shad Storhaug (NightOwl888)
> >
> > From: Shad Storhaug
> > Sent: Tuesday, March 21, 2017 5:25 AM
> > To: ''
> > Subject: API Work/Stabilization Update
> >
> > I am getting very close to getting #203 merged. I wouldn't go so far 
> > as to say that the API is finished, but the most significant of the 
> > breaking API changes are now behind us.
> >
> >
> > I just wanted to be sure there is someone available to help get the 
> > build working after the merge. I think it would be appropriate to 
> > change the pre-release label from "beta" to "beta2" (without 
> > resetting the build number, since that is actually what NuGet uses). 
> > This would be primarily because of a major breaking API change, but 
> > also to indicate another advancement toward release.
> >
> > We should probably also get this onto NuGet as soon as possible to
> > (hopefully) make it easier to recruit help to stabilize and create 
> > some integration packages for popular Microsoft frameworks.
> >
> >
> >
> > 1.       The QueryParser.Flexible custom localized message functionality
> > is currently not implemented for .NET core, so those tests are now
> failing.
> >
> > 2.       The implementation of Lucene.Net.Expressions currently reads
> data
> > from the configuration file. This is not how modern libraries are 
> > supposed to be built - instead we want any configuration to be 
> > pushed in from the application that uses Lucene.Net. Reading from 
> > the configuration file directly means no opportunity to use 
> > dependency injection. There is also a namespace 
> > Support/Configuration that can and should be removed after the 
> > implementation is refactored to be DI-friendly (see 
> > I haven't 
> > yet worked out how the implementation was done in .NET - in Java, 
> > the defaults were read from an embedded resource file and could be 
> > overridden by passing in a ClassLoader (similar to .NET's Assembly
> class) - if anyone has any information on how the "auto generated" C# 
> code was generated, please share.
> >
> > 3.       The Collation functionality in Analysis.Common doesn't work with
> > icu-dotnet, and has been excluded from compilation using the 
> > constant FEATURE_COLLATION. I am now convinced after reading the 
> > docs that it would be better to port the similar functionality from 
> > Analysis.ICU because it was designed to work with icu4j and is 
> > therefore more likely to work with icu-dotnet.
> >
> > 4.       The Highlighter PostingsHighlight and VectorHighlight
> > functionality relies on icu-dotnet, which doesn't have a close match 
> > for the BreakIterator in the JRE, so there are likely some big 
> > differences in the functionality. Several hacks were put in to make 
> > the tests pass, but these are not likely to fix all of the issues in 
> > the
> wild.
> >
> > 5.       There are several namespaces in Lucene.Net.Core and
> > Lucene.Net.Codecs that have broken documentation comments.
> >
> > 6.       There are some concurrency and performance issues (as pointed
> out
> > by Vincent Van Den Burghe):
> >
> >
> > 7.       We have around 2 dozen tests that fail during randomization
> > (averaging about 17 broken per run), and 8 tests that fail all/most 
> > of the time.
> >
> > RESOLVED ISSUES (in addition to API refactoring)
> >
> >
> > 1.       Finished implementing the randomization of Codecs, Culture, Time
> > Zone, and InfoStream in the TestFramework.
> >
> > 2.       Added factories for Codec, DocValuesFormat, and PostingsFormat
> so
> > custom implementations can be provided via dependency injection 
> > instead of using the Java-ish NamedSPILoader class. The name must 
> > now be provided by an attribute (or by class naming convention) 
> > rather than via constructor, so it can be read without creating an 
> > instance of
> the class.
> >
> > 3.       Fixed several of the codecs in Lucene.Net.Codecs that were still
> > not functioning (and not being tested because of the unfinished 
> > RandomCodec class and test mocks).
> >
> > 4.       Reviewed all catch blocks in Lucene.Net.Core to ensure the right
> > type of exceptions are being caught and the right type re-thrown.
> >
> > 5.       Fixed culture-sensitive comparison and sort order issues when
> > using strings in Lucene.Net.Core and Lucene.Net.Codecs.
> >
> > 6.       Merged similar functionality in Support into the same class and
> > deleted several unused Support classes.
> >
> > 7.       Made the API CLS compliant, so it now works with all .NET
> > languages.
> >
> >
> > Shad Storhaug (NightOwl888)
> >
View raw message