hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Regionservers consuming too much ram in HDP 2.6.
Date Fri, 08 Dec 2017 12:08:56 GMT
Thanks Esteban.

Probably Esteban is right. I have not seen a case where we have configured
1G but RS alone occupies 16G in the RAM.  so my knowledge in this case
could be limited here.

Yes the important thing is does the RS OOME. If that is not the case then
cluster is operable and does not go for aborts.

Regards
Ram




On Thu, Dec 7, 2017 at 9:40 PM, Esteban Gutierrez <esteban@cloudera.com>
wrote:

> Hi Lalit,
>
> I don't think there shouldn't be any concern for you about RAM used by the
> JVM as long as it stays around the heap size + few 100s of MBs for stack
> and DM as Ram mentioned. Even if there is no activity on HBase (no puts, no
> reads) objects will be created and destroyed from time to time and the JVM
> will be touching RAM pages during this process and  the OS will be marking
> those memory pages as used even if they have been GC'd by the JVM. As long
> as you don't see OOMEs in the JVM or running low of available memory in the
> JVM there is no issue.
>
> thanks,
> esteban.
>
>
> --
> Cloudera, Inc.
>
>
> On Thu, Dec 7, 2017 at 2:43 AM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
> > Your JDK parameters and configs. Heap configs set in hbase-env.sh etc.
> > The hbase-site.xml details reltaed to memstore and cache.
> >
> > Regards
> > Ram
> >
> > On Thu, Dec 7, 2017 at 12:07 PM, Lalit Jadhav <
> lalit.jadhav@nciportal.com>
> > wrote:
> >
> > > Hello,
> > >
> > > Yes, it is an interesting case.
> > > Ram, Sorry I cannot share whole details, but let me know params you
> need.
> > >
> > >
> > > On Thu, Dec 7, 2017 at 11:58 AM, ramkrishna vasudevan <
> > > ramkrishna.s.vasudevan@gmail.com> wrote:
> > >
> > > > I don have any idea now from this. Seems to be an interesting case
> > since
> > > > none of the users have reported such an issue. I need to try out this
> > to
> > > > ascertain what is really happening.
> > > >
> > > > Can you paste your hbase-env.sh details and the hbase-site.xml just
> to
> > be
> > > > sure there is nothing overriding the configs?
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > >
> > > > On Wed, Dec 6, 2017 at 5:41 PM, Lalit Jadhav <
> > lalit.jadhav@nciportal.com
> > > >
> > > > wrote:
> > > >
> > > > > Are you sure the 16G taken up in the RAM is due to the region
> server?
> > > > > :Yes
> > > > >
> > > > > Are you having any other cache configuration like  bucket cache?
> > > > > :No
> > > > >
> > > > > Are you allocating any Direct_memory for the region servers?
> > > > > :No
> > > > >
> > > > > So when does this raise to 16G happen - is it after the regions are
> > > > created
> > > > > or even before them i.e just when you start the region server?
> > > > > : Even before regions are created.
> > > > >
> > > > > Which version of hbase is it?
> > > > > 1.1.2
> > > > >
> > > > > On Dec 6, 2017 4:02 PM, "ramkrishna vasudevan" <
> > > > > ramkrishna.s.vasudevan@gmail.com> wrote:
> > > > >
> > > > > > Few more questions,
> > > > > > Are you sure the 16G taken up in the RAM is due to the region
> > server?
> > > > Are
> > > > > > you having any other cache configuration like  bucket cache?
> > > > > > Are you allocating any Direct_memory for the region servers?
> > > > > >
> > > > > > So when does this raise to 16G happen - is it after the regions
> are
> > > > > created
> > > > > > or even before them i.e just when you start the region server?
> > Which
> > > > > > version of hbase is it?
> > > > > >
> > > > > > Regards
> > > > > > Ram
> > > > > >
> > > > > > On Wed, Dec 6, 2017 at 3:57 PM, Lalit Jadhav <
> > > > lalit.jadhav@nciportal.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Ramkrishna,
> > > > > > >
> > > > > > > Thanks for reply,
> > > > > > > Right now I am not performing any operation on HBase(it
is
> idle),
> > > > > Still,
> > > > > > > utilization is 16GB. But when I shut down them, It frees
this
> > > memory.
> > > > > > >
> > > > > > > On Wed, Dec 6, 2017 at 2:41 PM, ramkrishna vasudevan <
> > > > > > > ramkrishna.s.vasudevan@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi Lalith
> > > > > > > >
> > > > > > > > Seems you have configured very minimum heap space.
> > > > > > > > So when you say RAM size is increasing  what are the
> operations
> > > > that
> > > > > > you
> > > > > > > > are performing when the memory increases. I can see
heap is
> > only
> > > 1G
> > > > > but
> > > > > > > > still your  memory is 16G.Are you sure that 16G is
only due
> to
> > > > Region
> > > > > > > > servers? Are you having heavy writes or reads during
that
> time.
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > Ram
> > > > > > > >
> > > > > > > > On Wed, Dec 6, 2017 at 2:31 PM, Lalit Jadhav <
> > > > > > lalit.jadhav@nciportal.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Adding More info,
> > > > > > > > >
> > > > > > > > > Regions per region server : 13-15 Regions per
RS
> > > > > > > > >
> > > > > > > > > memory region server is taking : 16GB
> > > > > > > > >
> > > > > > > > > Memstore size : 64 MB
> > > > > > > > >
> > > > > > > > > configured heap for Master : 1 GB
> > > > > > > > >
> > > > > > > > > configured heap for RegionServer : 1 GB
> > > > > > > > >
> > > > > > > > > On Tue, Dec 5, 2017 at 5:40 PM, Lalit Jadhav
<
> > > > > > > lalit.jadhav@nciportal.com
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello All,
> > > > > > > > > >
> > > > > > > > > > When we do any operations on the database(HBase),
> > > RegionServers
> > > > > are
> > > > > > > > > taking
> > > > > > > > > > too much ram and not releases until we restart
them. Is
> > there
> > > > any
> > > > > > > > > parameter
> > > > > > > > > > or property to release ram or to restrict
RegionServers
> to
> > > take
> > > > > > this
> > > > > > > > much
> > > > > > > > > > of memory? Help will be appreciated.
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Regards,
> > > > > > > > > > Lalit Jadhav
> > > > > > > > > > Network Component Private Limited.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Regards,
> > > > > > > > > Lalit Jadhav
> > > > > > > > > Network Component Private Limited.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Regards,
> > > > > > > Lalit Jadhav
> > > > > > > Network Component Private Limited.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Lalit Jadhav
> > > Network Component Private Limited.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message