hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ayon Sinha <ayonsi...@yahoo.com>
Subject Re: Data migration in Hadoop
Date Tue, 13 Sep 2011 17:50:59 GMT
What you can do for each node:
1. decommission node (or 2 nodes if you want to do this faster). You can do this with the
excludes file.
2. Wait for blocks to be moved off the decommed node(s)
3. Replace the disks and put them back in service.
4. Repeat until done.
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.



________________________________
From: Vikas Srivastava <vikas.srivastava@one97.net>
To: user@hive.apache.org
Sent: Tuesday, September 13, 2011 5:27 AM
Subject: Re: Data migration in Hadoop


hey sonal!!

Actually right now we have 11 node cluster each having 8 disks of 3oogb and 8gb ram,

now what we want to do is to replace those 300gb disks with 1 tb disks so that we can have
more space per server.

we have replication factor 2.

my suggestion is ..
1:- Add a node of 8 tb in cluster and run balancer to balance the load.
2:- free any 1 node(repalcement node)..... 

question:- does the imbalance size in cluster is of any datanode create a problem...or have
any bad impact

regards 
Vikas Srivastava



On Tue, Sep 13, 2011 at 5:37 PM, Sonal Goyal <sonalgoyal4@gmail.com> wrote:

Hi Vikas,
>
>
>This was discussed in the groups recently:
>
>
>http://lucene.472066.n3.nabble.com/Fixing-a-bad-HD-tt2863634.html#none
>
>
>Are you looking at replacing all your datanodes, or only a few? how big is your cluster?
>
>Best Regards,
>Sonal
>Crux: Reporting for HBase
>Nube Technologies 
>
>
>
>
>
>
>
>
>
>On Tue, Sep 13, 2011 at 1:52 PM, Vikas Srivastava <vikas.srivastava@one97.net> wrote:
>
>HI ,
>>
>>can ny1 tell me how we can migrate hadoop or replace old hard disks with new big size
hdd.
>>
>>actually i need to replace old hdd of 300 tbs to 1 tb so how can i do this efficiently!!!
>>
>>ploblem is to migrate data from 1 hdd to other
>>
>>
>>-- 
>>With Regards
>>Vikas Srivastava
>>
>>DWH & Analytics Team
>>Mob:+91 9560885900
>>One97 | Let's get talking !
>>
>


-- 
With Regards
Vikas Srivastava

DWH & Analytics Team
Mob:+91 9560885900
One97 | Let's get talking !
Mime
View raw message