lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@cs.put.poznan.pl>
Subject Re: test buffering causes OOM errors
Date Mon, 23 Jul 2012 08:45:44 GMT
Just for completeness -- something I forgot about.

There is one more option that I forgot about -- that <junit4> ANT
target is meant to simplify splitting suites into multiple JVMs and
providing nicer integration with the randomized runner (seed passing
etc). You can still use a regular ANT's <junit> to run the tests
(sequentially) and this will provide you with an immediate pipe of the
test's output to the console. This doesn't mean that it won't OOM on
large outputs because test's outputs are passed as Strings to report
instances (so it will OOM sooner or later).

D.

On Sun, Jul 22, 2012 at 8:53 AM, Dawid Weiss
<dawid.weiss@cs.put.poznan.pl> wrote:
>> +1, I also miss simple streamed output option when running a test.
>>
>> And it's fine if it only works if you run one test, one JVM, etc.
>
> There is an issue for 1-jvm streaming on github. Like I said -- it is
> possible to decrease memory consumption (stdout buffering)  although
> there will always be limitations (like if you run
> -Dtests.iters=10000000 this results in 10000000 physical instances of
> a Description object that need to be at once in memory; JUnit
> architecture -- nothing I can do about it).
>
> I will get back to this when I come back from vacation. In the
> meantime I think I provided enough workarounds so that you can
> continue working? I mean Solr mostly uses java logging and redirecting
> this to a file is trivial. Mike -- you can do the same thing if you
> need tons of logs. There is also an option of doing:
>
> ANT_OPTS="-Xmx100g"
>
> or whatever amount of memory you have -- this is crappy in the sense
> that you'll see the output only after the test finishes but it also
> should work in the mean time until I do that refactoring.
>
> I admit harnessing threading was my priority and I am pretty happy
> with the result although time will tell how usefl There are nuances
> and blockers from doing this right resulting from JUnit infrastructure
> that are mind boggling, eh, long story.
>
> Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message