hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: hbase set memstore size to 0
Date Sun, 03 Aug 2014 19:48:55 GMT
Just to add to Kevin's comments. Are you looking at that because of the
split policy? If that's the case, can you not just change the policy? Like
configure your table to use the ConstantSizeRegionSplitPolicy?


2014-08-03 15:15 GMT-04:00 Kevin O'dell <kevin.odell@cloudera.com>:

> Upon insert, lower the global setting not the flush size :)
> On Aug 3, 2014 3:01 PM, "ozhang" <ozhangulen@gmail.com> wrote:
>
> > Hi Kevin,
> > We guess that, on region server start up, hbase gets some memory for each
> > memstore. So we want to decrease this value. You are saying that memstore
> > size doesnt matter. If memstore flush size 128 mb, does java take some
> > memory for each memstore on region startup. Or it only takes memory while
> > you are using it to insert data.
> > Thanks a lot
> > 3 Ağu 2014 21:27 tarihinde "Kevin O'dell [via Apache HBase]" <
> > ml-node+s679495n4062260h26@n3.nabble.com> yazdı:
> >
> > > Hi Ozhang,
> > >
> > >   If you are only bulk loading into HBase, then memstore flush size
> > should
> > > not matter. You most likely you looking to lower the upper/global
> > memstore
> > > limits.
> > > On Aug 3, 2014 2:23 PM, "ozhang" <[hidden email]
> > > <http://user/SendEmail.jtp?type=node&node=4062260&i=0>> wrote:
> > >
> > > > Hello,
> > > > In our hbase cluster memstore flush size is 128 mb. And to insert
> data
> > > to
> > > > tables, we only use bulk load tool. Since bulk loading bypasses
> > > memstores,
> > > > they are never used, so we want to minimize memstore flush size. But
> > > > memstore flush size is used in many important calculations in hbase
> > such
> > > > that;
> > > >
> > > > region split size = Min (R^2 * “hbase.hregion.memstore.flush.size”,
> > > > “hbase.hregion.max.filesize”)
> > > >
> > > > So setting memstore value smaller or "0" for example,  results in
> some
> > > > other
> > > > problems.
> > > > What do you suggest us in that case. Setting memstore size to 128
> holds
> > > > some
> > > > memory for tens of regions in region server and we want to get rid of
> > > it.
> > > > Thanks a lot.
> > > >
> > > > ozhan
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> >
> http://apache-hbase.679495.n3.nabble.com/hbase-set-memstore-size-to-0-tp4062257.html
> > > > Sent from the HBase User mailing list archive at Nabble.com.
> > > >
> > >
> > >
> > > ------------------------------
> > >  If you reply to this email, your message will be added to the
> discussion
> > > below:
> > >
> > >
> >
> http://apache-hbase.679495.n3.nabble.com/hbase-set-memstore-size-to-0-tp4062257p4062260.html
> > >  To unsubscribe from hbase set memstore size to 0, click here
> > > <
> >
> http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4062257&code=b3poYW5ndWxlbkBnbWFpbC5jb218NDA2MjI1N3wxMDY0OTA3MTE4
> > >
> > > .
> > > NAML
> > > <
> >
> http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> > >
> > >
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-hbase.679495.n3.nabble.com/hbase-set-memstore-size-to-0-tp4062257p4062261.html
> > Sent from the HBase User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message