Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.220.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <201405231053102287376@neusoft.com>
References: <201405221320007232532@neusoft.com>
	<CAPQV63XRhhyTAcawd9yaOH0zmmqkFFQuKCwuT5hNxmO7Mfe0xA@mail.gmail.com>
	<201405231038402628135@neusoft.com>
	<A0416409-AB7D-49A2-B66A-2CDFFC22A6A6@gmail.com>
	<201405231053102287376@neusoft.com>
Date: Thu, 22 May 2014 19:59:32 -0700
Message-ID: 
 <CALte62zvKRFc-17QkqnB+-V2BswQUjOB15MM7QTBjq1AUg6JqA@mail.gmail.com>
Subject: Re: Re: CDH5 hbase client outofmemory
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>,
 jingych <jingych@neusoft.com>
Content-Type: multipart/alternative; boundary=20cf302ef54439301304fa086bfd

--20cf302ef54439301304fa086bfd
Content-Type: text/plain; charset=UTF-8

There're many options.

https://*imgur*.com/ is one.

Cheers


On Thu, May 22, 2014 at 7:53 PM, jingych <jingych@neusoft.com> wrote:

> Sorry. This is a image. I do not know where can i upload it.
>
> Could you please give me some suggestions.
>
>
>
>
> jingych
>
> From: Ted Yu
> Date: 2014-05-23 10:45
> To: user@hbase.apache.org
> CC: Jean-Marc Spaggiari; user
> Subject: Re: CDH5 hbase client outofmemory
> The snapshot didn't go through.
>
> Can you put it on some website and give us the URL ?
>
> Cheers
>
> On May 22, 2014, at 7:38 PM, jingych <jingych@neusoft.com> wrote:
>
> > Hi, JM & everyone!
> >
> > Thanks for your reply!
> >
> > My java client, use the HTable#put(Put) method with 2M buffer to commit
> rows.
> > While the client running for a while, the jvisualvm shows the htable
> threads increasing and the heap size increased too.
> > Here is the snapshot:
> >
> >
> > i research the implementation of the HTable#put(Put).
> > I found the doPut operation is the async, and when one row success, this
> will return.
> >
> > So, I guess when the HBase server is slower to response the put request.
> > The hbase client will accumulate so many rows.
> > And on the other hand, when the put operation failed, the hbase client
> will retry with the recursion submit method.
> >
> > I think the above two is the reason caused the OOME.
> >
> >
> > PS:
> > What do you mean It's not cdh5 specific? I found it from the website
> http://archive.cloudera.com/cdh5/cdh/5/hbase-0.96.1.1-cdh5.0.0/
> > Where can i get the CDH5 source code ?
> > CDH5 means hbase 0.96.2?
> > I'm confused with the CDH5, Apache, Phoenix ...
> > What's the relationship among them.
> >
> > Thanks a lot!
> >
> > Best regards!
> >
> > Jingych
> >
> > From: Jean-Marc Spaggiari
> > Date: 2014-05-23 09:33
> > To: user; jingych
> > Subject: Re: CDH5 hbase client outofmemory
> > Hi Jingych,
> >
> > This is the HBase 0.96.2 code. Not CDH5 specific. Do you have more
> details on you OOME? Did you figured why it occured?
> >
> > JM
> >
> >
> > 2014-05-22 1:20 GMT-04:00 jingych <jingych@neusoft.com>:
> >> Hello, everyone!
> >>
> >> I found the CDH5 hbase client swallowed the Outofmemory exception.
> >>
> >> It didn't throw out, cause the program couldn't make the correct
> response to process the OOM.
> >>
> >> Is't good to catch the client OOM? Why?
> >>
> >> The CDH5 hbase client process the throwable code:
> >> private boolean manageError(int originalIndex, Row row, boolean
> canRetry,
> >>                               Throwable throwable, HRegionLocation
> location) {
> >>     if (canRetry && throwable != null && throwable instanceof
> DoNotRetryIOException) {
> >>       canRetry = false;
> >>     }
> >>
> >>     byte[] region = null;
> >>     if (canRetry && callback != null) {
> >>       region = location == null ? null :
> location.getRegionInfo().getEncodedNameAsBytes();
> >>       canRetry = callback.retriableFailure(originalIndex, row, region,
> throwable);
> >>     }
> >>
> >>     if (!canRetry) {
> >>       if (callback != null) {
> >>         if (region == null && location != null) {
> >>           region = location.getRegionInfo().getEncodedNameAsBytes();
> >>         }
> >>         callback.failure(originalIndex, region, row, throwable);
> >>       }
> >>       errors.add(throwable, row, location);
> >>       this.hasError.set(true);
> >>     }
> >>
> >>     return canRetry;
> >>   }
> >>
> >> This will treat the OOM as the canRetry exception.
> >>
> >> Thanks!
> >>
> >> Best Regards!
> >>
> >>
> >>
> >>
> >> jingych
> >>
> ---------------------------------------------------------------------------------------------------
> >> Confidentiality Notice: The information contained in this e-mail and
> any accompanying attachment(s)
> >> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> >> Neusoft Corporation, its subsidiaries and/or its affiliates. If any
> reader of this communication is
> >> not the intended recipient, unauthorized use, forwarding, printing,
>  storing, disclosure or copying
> >> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> >> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> >> your system. Thank you.
> >>
> ---------------------------------------------------------------------------------------------------
> >
> >
> ---------------------------------------------------------------------------------------------------
> > Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> > is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> > Neusoft Corporation, its subsidiaries and/or its affiliates. If any
> reader of this communication is
> > not the intended recipient, unauthorized use, forwarding, printing,
>  storing, disclosure or copying
> > is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> > immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> > your system. Thank you.
> >
> ---------------------------------------------------------------------------------------------------
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
>  storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>

--20cf302ef54439301304fa086bfd--