hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steven zhuang <steven.zhuang.1...@gmail.com>
Subject Re: why DFSClient.BlockReader doesnot support seek(long) can I use skip() instead?
Date Tue, 21 Sep 2010 02:08:32 GMT
thanks, man,
these small files will be frequently accessed, our idea is that by
caching data location from namenode, we may relieve  the load of
NameNode and maybe also save sometime at network communication.


On Tue, Sep 21, 2010 at 1:10 AM, Allen Wittenauer
<awittenauer@linkedin.com> wrote:
>
> On Sep 19, 2010, at 7:57 PM, steven zhuang wrote:
>
>> hi, all,
>>       I have sent this mail in common user list before, duplicate it
>> here to seek for more help from experts.
>
> You'll likely have more luck on hdfs-dev.
>
>>       I am wondering why seek(long) is disabled in HDFS.BlockReader?
>> Can I use skip(long) to replace this seek(long)?
>>
>>       I have a bunch of small files, each is less than a block in size.
>
>        In other words, using Hadoop against recommended best practices. :)
>
>
>> In
>> my program, given the file/block information, I will try to start a
>> process on each datanode and try to read from the HDFS directly
>> through a socket connection to the datanode.
>
>        Again, using Hadoop against best practices.
>
>>      The read requires seek OP on the file, cause the file I used is
>> TFile, which requires the underlying class to be seekable.
>
>

Mime
View raw message