hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2172) PositionCache was removed from FSDataInputStream, causes extremely bad MapFile performance
Date Fri, 09 Nov 2007 17:59:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541389
] 

Hairong Kuang commented on HADOOP-2172:
---------------------------------------

Yes, DFS and any ChecksumFileSystem like LocalFileSystem all already cache file poition somewhere.
That's why I took out CachedPoisition in FSDataInputStream. The problem is caused by RawLocalFileSystem
not caching its file position.

Doug and I talked over IM and he thinks it is better to put the cached file position in the
highest level as possible so it can be shared. I feel it is better cached in BufferedFSInputStream.

> PositionCache was removed from FSDataInputStream, causes extremely bad MapFile performance
> ------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-2172
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2172
>             Project: Hadoop
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.14.3, 0.15.0
>            Reporter: Johan Oskarsson
>            Assignee: Johan Oskarsson
>            Priority: Blocker
>             Fix For: 0.15.1
>
>         Attachments: HADOOP-2172-2.patch, HADOOP-2172-3.patch, positioncache-v1.patch
>
>
> The PositionCache in FSDataInputStream seems to have been removed in HADOOP-1470. This
causes for example MapFile.get usage to be  extremely slow as the file position isn't cached
in memory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message