hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Mor <amit.mor.m...@gmail.com>
Subject Re: Calculating size in HBase.
Date Fri, 05 Jul 2013 11:24:55 GMT
>From hadooo shell man

Usage: hadoop fs -du URI [URI …]

Displays aggregate length of files contained in the directory or the length
of a file in case its just a file. Example: hadoop fs -du /user/hadoop/dir1
/user/hadoop/file1 hdfs://nn.example.com/user/hadoop/dir1 Exit Code:
Returns 0 on success and -1 on error.

dus Usage: hadoop fs -dus <args>

Displays a summary of file lengths."

You just have to supply your hbase' hdfs path
On Jul 5, 2013 6:55 AM, "Bikash Agrawal" <er.bikash21@gmail.com> wrote:

> Hi ,
> Need to calculate data size in HBase. I can do it by using KV length. But
> it is time consuming with huge data block.
> HFile looks better solution but still I have question with HFile, HFile
> will give size of data block. What if I have limit in scan[ ] object start
> row and end row. And those range can be some where in two different HFile.
> To calculate exact size I need to use loop, which is again inefficient.
> Actually, I am trying to run map reduce using HBase as source. I need to
> know what is data size. I am trying to do some performance evaluation.
> Still I can do some approximation calculation, looking at map-reduce input.
> It is always better to know exact size.
> Do you know how can do calculate data size in HBASE.
> --
> With Best Regards
> Bikash Agrawal
> Web/Software Developer
> Mobile: +47 92502701
> www.bikashagrawal.com.np

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message