Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of asaf.mesika@gmail.com
 designates 209.85.214.54 as permitted sender)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1283)
Subject: Re: HDFS disk space requirements
From: "Mesika, Asaf" <asaf.mesika@gmail.com>
In-Reply-To: 
 <CA+NDPeed+Mr9UL2mi+98h92FgCYHog3OPNzOj7yUocr-ttmaYw@mail.gmail.com>
Date: Fri, 11 Jan 2013 12:23:27 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <DFFE3599-CE5D-4DB4-943F-60ED49BA7E46@gmail.com>
References: 
 <CA+NDPeed+Mr9UL2mi+98h92FgCYHog3OPNzOj7yUocr-ttmaYw@mail.gmail.com>
To: user@hbase.apache.org

130 GB raw data will take in HBase since it adds the family name, =
qualifier and timestamp to each value, so it can even be 150GB. You can =
check it exactly, by loading only one row with one column and see how =
much it takes on the HDFS file system (run compaction first).

Next, you 5 times that since you have 5 times replication, so =
5x150=3D750GB

On Jan 11, 2013, at 5:07 AM, Panshul Whisper wrote:

> Hello,
>=20
> I have a 5 node hadoop cluster and a fully distributed Hbase setup on =
the
> cluster with 130 GB of HDFS space avaialble. HDFS replication is set =
to 5.
>=20
> I have a total of 115 GB of JSON files that need to be loaded into the
> Hbase database and then they have to processed.
>=20
> So is the available HDFS space sufficient for the operations?? =
considering
> the replication and all factors?
> or should I increase the space and by how much?
>=20
> Thanking You,
>=20
> --=20
> Regards,
> Ouch Whisper
> 010101010101