incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: [jira] [Commented] (BLUR-85) Kill JVM on OutOfMemory Error
Date Sat, 18 May 2013 17:27:53 GMT
I think that setting the memory very small is fine, it's a tough setting to
test in a real test.  The biggest issue this solves is the hung JVM that is
throwing OOM errors but the process is not exiting.  This can cause an
entire cluster to become unresponsive.  To be honest since this is a change
to the settings on the JVM, I'm fine with it the way it's implemented and
tested now.  Because I know the JVM will fire the command on OOM error.

Aaron


On Sat, May 18, 2013 at 1:19 PM, Gagan Juneja <gagandeepjuneja@gmail.com>wrote:

> Aaron,
> I have committed these changes. But one question how can we test this
> scenario. Do you have any specific case or we should simulate this
> issue by just setting memory value to be very small.
>
> Regards,
> Gagan
>
> On Sat, May 18, 2013 at 9:54 PM, Aaron McCurry <amccurry@gmail.com> wrote:
> > Yep I think that's its as simple as that.  I didn't know %p would give
> you
> > the PID of the jvm.  Cool, Thanks!
> >
> > Aaron
> >
> >
> > On Sat, May 18, 2013 at 2:31 AM, Gagan Juneja <gagandeepjuneja@gmail.com
> >wrote:
> >
> >> I think -XX:OnOutOfMemoryError="kill -9 %p" will automatically detects
> >> the pid. Do we need to pass it explicitly or %p would be sufficient.
> >>
> >>
> >> Regards,
> >> Gagan
> >>
> >> On Wed, May 15, 2013 at 11:37 PM, Aaron McCurry (JIRA) <jira@apache.org
> >
> >> wrote:
> >> >
> >> >     [
> >>
> https://issues.apache.org/jira/browse/BLUR-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13658614#comment-13658614
> ]
> >> >
> >> > Aaron McCurry commented on BLUR-85:
> >> > -----------------------------------
> >> >
> >> > Yep, that sounds right.  We should also add to the
> >> BLUR_SHARD_JVM_OPTIONS.  We should issue a kill -9 on the process and
> you
> >> can get the pid from the ./pids directory.  Take a look at the
> >> startup/shutdown scripts to get the exact file format.  I think it's
> >> "shard-0.pid" for the first shard process on the computer and
> >> "controller-0.pid" for the first controller and so on for the 2nd and
> third
> >> processes (if any).  Thanks!
> >> >
> >> > Aaron
> >> >
> >> >> Kill JVM on OutOfMemory Error
> >> >> -----------------------------
> >> >>
> >> >>                 Key: BLUR-85
> >> >>                 URL: https://issues.apache.org/jira/browse/BLUR-85
> >> >>             Project: Apache Blur
> >> >>          Issue Type: Improvement
> >> >>    Affects Versions: 0.1.5
> >> >>            Reporter: Aaron McCurry
> >> >>             Fix For: 0.1.5
> >> >>
> >> >>
> >> >> Most of the time if a Blur process runs out of memory it will likely
> >> hang and never recover.  So on an OOM error we should use the
> >> -XX:OnOutOfMemoryError="<cmd args>; <cmd args>" JVM option to kill
the
> >> process.
> >> >
> >> > --
> >> > This message is automatically generated by JIRA.
> >> > If you think it was sent incorrectly, please contact your JIRA
> >> administrators
> >> > For more information on JIRA, see:
> >> http://www.atlassian.com/software/jira
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message