hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonal Goyal <sonalgoy...@gmail.com>
Subject Re: Data migration in Hadoop
Date Wed, 14 Sep 2011 02:01:22 GMT
Vikas,

I would suggest running the production clusters with replication factor of
3. Then you could decommission 2 nodes as Ayon suggests. Else one node at a
time.

Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>





On Tue, Sep 13, 2011 at 11:20 PM, Ayon Sinha <ayonsinha@yahoo.com> wrote:

> What you can do for each node:
> 1. decommission node (or 2 nodes if you want to do this faster). You can do
> this with the excludes file.
> 2. Wait for blocks to be moved off the decommed node(s)
> 3. Replace the disks and put them back in service.
> 4. Repeat until done.
>
> -Ayon
> See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
> Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>
>
> ------------------------------
> *From:* Vikas Srivastava <vikas.srivastava@one97.net>
> *To:* user@hive.apache.org
> *Sent:* Tuesday, September 13, 2011 5:27 AM
> *Subject:* Re: Data migration in Hadoop
>
> hey sonal!!
>
> Actually right now we have 11 node cluster each having 8 disks of 3oogb and
> 8gb ram,
>
> now what we want to do is to replace those 300gb disks with 1 tb disks so
> that we can have more space per server.
>
> we have replication factor 2.
>
> my suggestion is ..
> 1:- Add a node of 8 tb in cluster and run balancer to balance the load.
> 2:- free any 1 node(repalcement node).....
>
> question:- does the imbalance size in cluster is of any datanode create a
> problem...or have any bad impact
>
> regards
> Vikas Srivastava
>
>
> On Tue, Sep 13, 2011 at 5:37 PM, Sonal Goyal <sonalgoyal4@gmail.com>wrote:
>
> Hi Vikas,
>
> This was discussed in the groups recently:
>
> http://lucene.472066.n3.nabble.com/Fixing-a-bad-HD-tt2863634.html#none
>
> Are you looking at replacing all your datanodes, or only a few? how big is
> your cluster?
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
>
> On Tue, Sep 13, 2011 at 1:52 PM, Vikas Srivastava <
> vikas.srivastava@one97.net> wrote:
>
> HI ,
>
> can ny1 tell me how we can migrate hadoop or replace old hard disks with
> new big size hdd.
>
> actually i need to replace old hdd of 300 tbs to 1 tb so how can i do this
> efficiently!!!
>
> ploblem is to migrate data from 1 hdd to other
>
>
> --
> With Regards
> Vikas Srivastava
>
> DWH & Analytics Team
> Mob:+91 9560885900
> One97 | Let's get talking !
>
>
>
>
>
> --
> With Regards
> Vikas Srivastava
>
> DWH & Analytics Team
> Mob:+91 9560885900
> One97 | Let's get talking !
>
>
>
>

Mime
View raw message