hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: Non-sensical? Compaction Metrics Hbase UI
Date Mon, 11 Aug 2014 20:41:56 GMT
Unfortunately I don't know an easy way.  But I think you could figure this
out by combining logs and the above.  If you look at the logs on the server
compacting, it should show the number of files being compacted, the total
size, and the output file name.  Then watch for the .tmp file growing to
approximately the totalSize listed in the logs.

Maybe someone else knows of a fix to the actual problem or an easier way to
achieve this.


On Mon, Aug 11, 2014 at 4:31 PM, Geovanie Marquez <
geovanie.marquez@gmail.com> wrote:

> nice, just took a look at this path in my system. I see the growth but that
> would only apply to major_compactions where all files are picked up and
> compacted because minor compactions won't be trackable this way without
> knowing which files were selected by the compaction algorithm.
>
> is there a method for getting progress on these?
>
>
>
> On Mon, Aug 11, 2014 at 3:14 PM, Bryan Beaudreault <
> bbeaudreault@hubspot.com
> > wrote:
>
> > I've also seen that on 0.94.x.  The compaction progress is almost never
> > accurate.
> >
> > I've taken to doing:
> >
> > hdfs -du /hbase/<table>/<region>/.tmp
> >
> > to track the new hfile size, expecting it to be approximately the same
> size
> > as total hfiles pre-compaction.
> >
> >
> > On Mon, Aug 11, 2014 at 3:11 PM, Geovanie Marquez <
> > geovanie.marquez@gmail.com> wrote:
> >
> > > *Attribute Name* *Value*
> > > HBase Version 0.98.1-cdh5.1.0, rUnknown
> > > HBase Compiled Sat Jul 12 08:20:49 PDT 2014, jenkins
> > > Hadoop Version 2.3.0-cdh5.1.0,
> r8e266e052e423af592871e2dfe09d54c03f6a0e8
> > > Hadoop Compiled 2014-07-12T13:49Z, jenkins
> > >
> > >
> > > On Mon, Aug 11, 2014 at 12:56 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >
> > > > Which release of hbase are you using ?
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Mon, Aug 11, 2014 at 9:49 AM, Geovanie Marquez <
> > > > geovanie.marquez@gmail.com> wrote:
> > > >
> > > > > What is the meaning of the progress value below?
> > > > >
> > > > >
> > > > >
> > > > > I see the information broken down below in the following HBase UI.
> > > > >
> > > > >
> > > >
> > >
> >
> http://region-server-ip:60030/rs-status?filter=general#regionCompactStats
> > > > >
> > > > >
> > > > >
> > > > > Region Name Num.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> <tablename>,`\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1398130030457.2dddb2b81d70ee4b4aaf39ca830beb41.
> > > > >
> > > > >
> > > > >
> > > > > Compacting KVsNum.
> > > > >
> > > > > 13617
> > > > >
> > > > >
> > > > >
> > > > > Compacted KVsCompaction
> > > > >
> > > > > 9221037
> > > > >
> > > > >
> > > > >
> > > > > Progress
> > > > >
> > > > > 67717.09%
> > > > >
> > > > >
> > > > >
> > > > > Another example:
> > > > >
> > > > > Compacting KVsNum. Compacted KVsCompaction Progress
> > > > >
> > > > > 357792875 1624293960 453.98%
> > > > >
> > > > >
> > > > > The progress seems non-sensical. How should I interpret it?  I
> notice
> > > > > something more than 100% most of the time.
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message