lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oliver Albrecht <o.albre...@oliver-albrecht.de>
Subject RE: Stackoverflow exception
Date Tue, 08 Aug 2017 08:08:27 GMT
Hello Itamar, hello Shad,

thanks for the fast response.

I could you provide the callstack, but i think it's useless because it's full of calls to
IsFinite(State s, OpenBitSet path, OpenBitSet visited) because it is a recursive function.
For the same reason i think that using the CI-Build wouldn't change anything. The IsFinite-function
is still working recursively.

I've tried to replace the recursion with a stack based approach (using Stack<State>),
but i'm not sure if my implementation is correct.

How ever, if i use my non-recursive version of IsFinite it crashes with a stackoverflow exception
in GetFiniteStrings(State s, HashSet<State> pathstates, HashSet<Int32sRef> strings,
Int32sRef path, int limit), which is also a recursive function. But this function is to complex
for me to convert it into a non-recursive version without exact knowledge what the function
should do.

In my opinion is the replacement of the recursion with a non-recursive approach the only solution.
Does no one else have this problem? I think to have an index with 4000 documents and a size
with 15 MB is not so extraordinary. Or is this only a problem how the suggester works?

I'm just try to use lucene to build a fulltext query engine for our internal dms system. The
system holds currently 450.000 documents with ca. 50 GB of data. I think the final index will
be around 2 GB of size.

kind regards 

Oliver

> Shad Storhaug <shad@shadstorhaug.com> hat am 7. August 2017 um 19:06 geschrieben:
> 
> Hi Oliver,
> 
> In addition to providing the full stack trace that Itamar mentioned, could you confirm
the problem still exists if you use the latest CI build here: [https://www.myget.org/gallery/lucene-net-ci?](https://www.myget.org/gallery/lucene-net-ci)
> 
> Thanks,
> Shad Storhaug (NightOwl888)
> 
> -----Original Message-----
> From: itamar.synhershko@gmail.com [mailto:itamar.synhershko@gmail.com] On Behalf Of Itamar
Syn-Hershko
> Sent: Monday, August 7, 2017 9:06 PM
> To: user@lucenenet.apache.org
> Subject: Re: Stackoverflow exception
> 
> What is the full stacktrace please?
> 
> --
> 
> Itamar Syn-Hershko
> Freelance Developer & Consultant
> Elasticsearch Partner
> Microsoft MVP | Lucene.NET PMC
> http://code972.com | @synhershko <https://twitter.com/synhershko> http://BigDataBoutique.co.il/
> 
> On Mon, Aug 7, 2017 at 4:49 PM, Oliver Albrecht < o.albrecht@oliver-albrecht.de>
wrote:
> 
> > Hello,
> > 
> > i'm using a DocumentDictionary to feed an AnalyzingSuggester using the
> > following code:
> > 
> > SnippetAnalyzingSuggester suggester = new AnalyzingSuggester(new
> > GermanAnalyzer(Lucene.Net.Util.LuceneVersion.LUCENE_48));
> > 
> > Lucene.Net.Store.Directory dir = Lucene.Net.Store.FSDirectory.
> > Open(indexDir);
> > 
> > IndexReader ir = DirectoryReader.Open(dir);
> > 
> > DocumentDictionary dict = new DocumentDictionary(ir, "Content", null,
> > null);
> > 
> > suggester.Build(dict.GetEntryIterator());
> > 
> > I get a stackoverflow exception on suggester.Build. The exception
> > throws in Lucene.Net.Util.Automaton.SpecialOperations.IsFinite.
> > 
> > The index contains 10.000 documents and has no payload and no weight.
> > 
> > Kind regards
> > 
> > Oliver

Mime
View raw message