drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hanifi Gunes <hgu...@maprtech.com>
Subject Re: forked process starves memory and dies
Date Thu, 14 May 2015 18:51:54 GMT
I was reverse reading the situation considering that Drill was starving and
exiting. It was OS killing the fork. Looks like memory is never been
returned back to OS causing starvation. Reducing max direct memory fixed
the problem.


On Wed, May 13, 2015 at 7:53 PM, Hanifi Gunes <hgunes@maprtech.com> wrote:

> I thought total, my bad. Any other idea why the forked process has been
> killed though?
>
> On Wed, May 13, 2015 at 6:33 PM, Jacques Nadeau <jacques@apache.org>
> wrote:
>
>> The 4gb is per fork, not total.
>>
>> On Wed, May 13, 2015 at 5:26 PM, Hanifi Gunes <hgunes@maprtech.com>
>> wrote:
>>
>> > I started constantly hitting this issue on my linux VM with 8 logical
>> > cores. What appears to be happing is forked process is being killed due
>> to
>> > memory starvation. I see that we are running surefire with max direct
>> > memory reservation of 4gb initially.
>> >
>> >
>> > On average 3G/8 forks makes 375M per fork and so per test suite. I would
>> > think that this should be good enough for many suites but do we have any
>> > test suites where we make huge allocations? Last but not least this may
>> > indicate we are leaking memory.
>> >
>> > *[INFO]
>> >
>> ------------------------------------------------------------------------*
>> >
>> > *[INFO] BUILD FAILURE*
>> >
>> > *[INFO]
>> >
>> ------------------------------------------------------------------------*
>> >
>> > *[INFO] Total time: 06:04 min*
>> >
>> > *[INFO] Finished at: 2015-05-14T00:07:26+00:00*
>> >
>> > *[INFO] Final Memory: 18M/380M*
>> >
>> > *[INFO]
>> >
>> ------------------------------------------------------------------------*
>> >
>> > *[ERROR] Failed to execute goal
>> > org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-cli)
>> on
>> > project drill-java-exec: ExecutionException: java.lang.RuntimeException:
>> > The forked VM terminated without properly saying goodbye. VM crash or
>> > System.exit called?*
>> >
>> >
>> > -Hanifi
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message