hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Agarwal <aagar...@hortonworks.com>
Subject Re: Copy hdfs block from one data node to another
Date Thu, 10 Jul 2014 04:34:06 GMT
The balancer does something similar. It uses
DataTransferProtocol.replaceBlock.


On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <sudhakara.st@gmail.com> wrote:

> You can get info about all blocks stored in perticuler data node, i,e
> block report. But you to handle, move in block level not in file or start
> and end bytes level.
>
>
> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <chris.mawata@gmail.com>
> wrote:
>
>> Haven't looked at the source but the thing you are trying to do sounds
>> similar to what happens when you are decommissioning a datanode. I would
>> hunt for that code.
>> Cheers
>> Chris
>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y.z.elshater@gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> Actually I need this functionality for my research, basically for fault
>>> tolerance. I can calculate some failure probability for some data nodes
>>> after certain unit of time. So I need to copy all the blocks reside on
>>> these nodes to another nodes.
>>>
>>> Thanks
>>> Yehia
>>>
>>>
>>> On 7 July 2014 20:45, Chris Mawata <chris.mawata@gmail.com> wrote:
>>>
>>>> Can you outline why one would want to do that? The blocks are
>>>> disposable so it is strange to manipulate them directly.
>>>>  On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y.z.elshater@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>> bytes) from one node to another node ?
>>>>>
>>>>> Thanks
>>>>> Yehia
>>>>>
>>>>
>>>
>
>
> --
>
> Regards,
> ...sudhakara
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message