hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YIMEN YIMGA Gael <gael.yimen-yi...@sgcib.com>
Subject RE: Need to evaluate a cluster
Date Thu, 10 Jul 2014 08:29:39 GMT

When I said Size of a disk is 3TB, this means that a datanode should have a disk space of
3TB (1 to 3 disk of 1TB to 3TB).

Could you please help me with your experience to approximate the number of nodes on one year


From: Oner Ak. [mailto:oak26013@gmail.com]
Sent: Wednesday 9 July 2014 21:31
To: user@hadoop.apache.org
Subject: Re: Need to evaluate a cluster

367 nodes sounded quite high for that amount of data per day. You might need 367 disks, but
do your nodes have more than one disk?

You may also take into account the compression factor that you are likely to use for the data
on the cluster.

9 Tem 2014 19:00 tarihinde "YIMEN YIMGA Gael" <gael.yimen-yimga@sgcib.com<mailto:gael.yimen-yimga@sgcib.com>>
Hello Dear,

I made an estimation of a number of nodes of a cluster that can be supplied by 720GB of data/day.
My estimation gave me 367 datanodes in a year. I’m a bit afraid by that amount of datanodes.
The assumptions, I used are the followings :

-          Daily supply (feed) : 720GB

-          HDFS replication factor: 3

-          Booked space for each disk outside HDFS: 30%

-          Size of a disk: 3TB.

I have two questions.

First, I would like to know if my assumptions are well taken?
Secondly, could someone help me to evaluate that cluster, to let me be sure that my results
are not to excessive, please ?

Standing by for your feedback

Warm regard

This message and any attachments (the "message") are confidential, intended solely for the
addressee(s), and may contain legally privileged information.
Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration.
Neither SOCIETE GENERALE nor any of its subsidiaries or affiliates shall be liable for the
message if altered, changed or
Please visit http://swapdisclosure.sgcib.com for important information with respect to derivative
Ce message et toutes les pieces jointes (ci-apres le "message") sont confidentiels et susceptibles
de contenir des informations couvertes
par le secret professionnel.
Ce message est etabli a l'intention exclusive de ses destinataires. Toute utilisation ou diffusion
non autorisee est interdite.
Tout message electronique est susceptible d'alteration.
La SOCIETE GENERALE et ses filiales declinent toute responsabilite au titre de ce message
s'il a ete altere, deforme ou falsifie.
Veuillez consulter le site http://swapdisclosure.sgcib.com afin de recueillir d'importantes
informations sur les produits derives.
View raw message