hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yehia Elshater <y.z.elsha...@gmail.com>
Subject Re: Copy hdfs block from one data node to another
Date Thu, 10 Jul 2014 06:51:28 GMT
Thanks a lot. I will take a look on the balancer and decommissioning code.


On 10 July 2014 00:34, Arpit Agarwal <aagarwal@hortonworks.com> wrote:

> The balancer does something similar. It uses
> DataTransferProtocol.replaceBlock.
>
>
> On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <sudhakara.st@gmail.com>
> wrote:
>
>> You can get info about all blocks stored in perticuler data node, i,e
>> block report. But you to handle, move in block level not in file or start
>> and end bytes level.
>>
>>
>> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <chris.mawata@gmail.com>
>> wrote:
>>
>>> Haven't looked at the source but the thing you are trying to do sounds
>>> similar to what happens when you are decommissioning a datanode. I would
>>> hunt for that code.
>>> Cheers
>>> Chris
>>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y.z.elshater@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Actually I need this functionality for my research, basically for fault
>>>> tolerance. I can calculate some failure probability for some data nodes
>>>> after certain unit of time. So I need to copy all the blocks reside on
>>>> these nodes to another nodes.
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>>
>>>> On 7 July 2014 20:45, Chris Mawata <chris.mawata@gmail.com> wrote:
>>>>
>>>>> Can you outline why one would want to do that? The blocks are
>>>>> disposable so it is strange to manipulate them directly.
>>>>>  On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y.z.elshater@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> How can copy a certain hdfs block (given the file name, start and
end
>>>>>> bytes) from one node to another node ?
>>>>>>
>>>>>> Thanks
>>>>>> Yehia
>>>>>>
>>>>>
>>>>
>>
>>
>> --
>>
>> Regards,
>> ...sudhakara
>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Mime
View raw message