lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@gmail.com>
Subject Re: test buffering causes OOM errors
Date Sat, 21 Jul 2012 10:51:52 GMT
I am on mobile but I will try to justify the design decisions I made later
on.

As for now, quickly, tests.iters has been fixed and should not oom unless
you run with many thousands of repetitions (if so it is better to loop
manually).

As for the oom on large sysouts - incease ant memory or redirect tests
logging (java logging subsystrm) to a file. Standard junit target from ant
suffers from the same issue really so this is not a regression over what
was available previously.

I think the easiest way, even for many repetitions, would be to redirect
the logging sink to a file and add a logger.info before each test so that
you can match failures with the log stream.

Sorry this comes so short but I'm on holidays.
On Jul 21, 2012 11:49 AM, "Yonik Seeley" <yonik@lucidimagination.com> wrote:

> On Sat, Jul 21, 2012 at 5:36 AM, Dawid Weiss <dawid.weiss@gmail.com>
> wrote:
> > Full stack trace would me more helpful
>
> Full stack trace below.
>
> > but I think the problem is that solr
> > tests emit megabytes of data to the console and this has to be buffered
> and
> > passed to report listeners.
>
> I've not looked into the test framework architecture and don't
> understand why things can't just be streamed - but from my perspective
> the most important thing is just getting the logs when a test fails.
> Logs are most useful in big stress tests where things are not easily
> reproducible or debuggable with a debugger - and these also tend to
> produce a lot of logging.
>
> A month or so ago, I also hit issues with tests.iters (I think I got
> OOM errors when I cranked it too high?)
> so I went back to looping the test myself in a shell script.
> I don't know what the exact behavior here is, but what would be most
> useful is if the output were reset for each run so that when things
> fail you are left with the full log for just the run that failed.
>
> > It is possible to spill to disk of course but not a trivial refactoring.
> >
> > An easy workaround for now would be to redirect java logging to a file
> and
> > keep the console output smaller.
> >
> > There is an issue for those Ooms on github but I admit it isn't of super
> > high priority for me given the workaround above?
>
> -Yonik
> http://lucidimagination.com
>
>
> [junit4:junit4] Event bus dispatch error: java.lang.OutOfMemoryError:
> Java heap space
> [junit4:junit4] java.lang.OutOfMemoryError: Java heap space
> [junit4:junit4]         at java.util.Arrays.copyOf(Arrays.java:2882)
> [junit4:junit4]         at
>
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
> [junit4:junit4]         at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
> [junit4:junit4]         at
> java.lang.StringBuffer.append(StringBuffer.java:224)
> [junit4:junit4]         at java.io.StringWriter.write(StringWriter.java:84)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.LineBufferWriter.write(LineBufferWriter.java:26)
> [junit4:junit4]         at java.io.Writer.write(Writer.java:96)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.PrefixedWriter.write(PrefixedWriter.java:32)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.PrefixedWriter.write(PrefixedWriter.java:41)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.flushOutput(WriterOutputStream.java:303)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.write(WriterOutputStream.java:219)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.write(WriterOutputStream.java:231)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.SlaveInfo.decodeStreams(SlaveInfo.java:129)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.TextReport.decodeStreamEvents(TextReport.java:480)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.TextReport.format(TextReport.java:443)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.TextReport.onSuiteResult(TextReport.java:249)
> [junit4:junit4]         at
> sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source)
> [junit4:junit4]         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4:junit4]         at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatingListener.receiveSuiteEnd(AggregatingListener.java:150)
> [junit4:junit4]         at
> sun.reflect.GeneratedMethodAccessor112.invoke(Unknown Source)
> [junit4:junit4]         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4:junit4]         at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
> [junit4:junit4] Event bus dispatch error: java.lang.OutOfMemoryError:
> Java heap space
> [junit4:junit4] java.lang.OutOfMemoryError: Java heap space
> [junit4:junit4]         at java.util.Arrays.copyOf(Arrays.java:2882)
> [junit4:junit4]         at
>
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
> [junit4:junit4]         at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
> [junit4:junit4]         at
> java.lang.StringBuffer.append(StringBuffer.java:306)
> [junit4:junit4]         at java.io.StringWriter.write(StringWriter.java:77)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.flushOutput(WriterOutputStream.java:303)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.write(WriterOutputStream.java:219)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.write(WriterOutputStream.java:231)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.SlaveInfo.decodeStreams(SlaveInfo.java:129)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.antxml.AntXmlReport.buildModel(AntXmlReport.java:149)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.antxml.AntXmlReport.onSuiteResult(AntXmlReport.java:95)
> [junit4:junit4]         at
> sun.reflect.GeneratedMethodAccessor103.invoke(Unknown Source)
> [junit4:junit4]         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4:junit4]         at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatingListener.receiveTestEnd(AggregatingListener.java:145)
> [junit4:junit4]         at
> sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
> [junit4:junit4]         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4:junit4]         at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.LocalSlaveStreamHandler.pumpEvents(LocalSlaveStreamHandler.java:174)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.LocalSlaveStreamHandler$1.run(LocalSlaveStreamHandler.java:94)
> [junit4:junit4] Event bus dispatch error: java.lang.OutOfMemoryError:
> Java heap space
> [junit4:junit4] java.lang.OutOfMemoryError: Java heap space
> [junit4:junit4]         at java.util.Arrays.copyOf(Arrays.java:2882)
> [junit4:junit4]         at
>
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
> [junit4:junit4]         at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
> [junit4:junit4]         at
> java.lang.StringBuffer.append(StringBuffer.java:306)
> [junit4:junit4]         at java.io.StringWriter.write(StringWriter.java:77)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.flushOutput(WriterOutputStream.java:303)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.processInput(WriterOutputStream.java:283)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.write(WriterOutputStream.java:214)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.org.apache.commons.io.output.WriterOutputStream.write(WriterOutputStream.java:231)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.json.JsonAggregatedSuiteResultEventAdapter.serializeEvents(JsonAggregatedSuiteResultEventAdapter.java:75)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.json.JsonAggregatedSuiteResultEventAdapter.serialize(JsonAggregatedSuiteResultEventAdapter.java:34)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.json.JsonAggregatedSuiteResultEventAdapter.serialize(JsonAggregatedSuiteResultEventAdapter.java:20)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.GsonToMiniGsonTypeAdapterFactory$3.write(GsonToMiniGsonTypeAdapterFactory.java:97)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:512)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.listeners.json.JsonReport.onSuiteResult(JsonReport.java:198)
> [junit4:junit4]         at
> sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source)
> [junit4:junit4]         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4:junit4]         at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatingListener.receiveTestEnd(AggregatingListener.java:145)
> [junit4:junit4]         at
> sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
> [junit4:junit4]         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit4:junit4]         at java.lang.reflect.Method.invoke(Method.java:597)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
> [junit4:junit4]         at
>
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264)
>

Mime
View raw message