hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Is it necessary to cache metadata in client side?
Date Fri, 11 Jun 2010 09:02:03 GMT
It is cached per input stream - see DFSInputStream.locatedBlocks,
prefetchSize, etc.


On Thu, Jun 10, 2010 at 11:43 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> Hi all,
> According the GFS paper claims, GFS will cache meta data in client.
> But when I check the source code of hadoop, it seems that hadoop won't
> cache it in client side. I just wan to make sure whether I am right ?
> And wondering whether there's someone work on it ? One advantage of
> caching metadata in client side I can think of is that tasktracker
> will fetch job.xml in HDFS. And most of time we will run multiple task
> in one node, so if tasktrack cache the metadata, it can reduce the
> communication with namenode.
> --
> Best Regards
> Jeff Zhang

Todd Lipcon
Software Engineer, Cloudera

View raw message