hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-200) In HDFS, sync() not yet guarantees data available to the new readers
Date Fri, 05 Feb 2010 22:16:31 GMT

    [ https://issues.apache.org/jira/browse/HDFS-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830327#action_12830327
] 

Todd Lipcon commented on HDFS-200:
----------------------------------

Hey Dhruba,

In the client-side length fetching code, you use check the LocatedBlock last = newInfo.get(newInfo.locatedBlockCount()-1)
to fetch the size.
However, this is only the last block in the file for cases where the file has fewer than prefetchSize
blocks, right? Has anyone tried a test
after setting block size to 1MB?

The trunk append code solves this issue by adding a lastBlock field to LocatedBlocks. I imagine
you avoided this to keep wire compatibility
in this patch.

-Todd

> In HDFS, sync() not yet guarantees data available to the new readers
> --------------------------------------------------------------------
>
>                 Key: HDFS-200
>                 URL: https://issues.apache.org/jira/browse/HDFS-200
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: dhruba borthakur
>            Priority: Blocker
>         Attachments: 4379_20081010TC3.java, fsyncConcurrentReaders.txt, fsyncConcurrentReaders11_20.txt,
fsyncConcurrentReaders12_20.txt, fsyncConcurrentReaders13_20.txt, fsyncConcurrentReaders14_20.txt,
fsyncConcurrentReaders15_20.txt, fsyncConcurrentReaders3.patch, fsyncConcurrentReaders4.patch,
fsyncConcurrentReaders5.txt, fsyncConcurrentReaders6.patch, fsyncConcurrentReaders9.patch,
hadoop-stack-namenode-aa0-000-12.u.powerset.com.log.gz, hdfs-200-ryan-existing-file-fail.txt,
hypertable-namenode.log.gz, namenode.log, namenode.log, Reader.java, Reader.java, reopen_test.sh,
ReopenProblem.java, Writer.java, Writer.java
>
>
> In the append design doc (https://issues.apache.org/jira/secure/attachment/12370562/Appends.doc),
it says
> * A reader is guaranteed to be able to read data that was 'flushed' before the reader
opened the file
> However, this feature is not yet implemented.  Note that the operation 'flushed' is now
called "sync".

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message