hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2757) Cannot read a local block that's being written to when using the local read short circuit
Date Sat, 08 Sep 2012 20:58:07 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins updated HDFS-2757:
------------------------------

    Attachment: hdfs-2757.txt

I thought both findbugs warnings were bogus but actually the 2nd one is due to this patch.
The access to locatedBlocks needs to be synchronized, in practice it should be fine, but findbugs
complains because all other accesses are synchronized. I pulled your check out to a private
synchronized method. Updated patch attached.
                
> Cannot read a local block that's being written to when using the local read short circuit
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-2757
>                 URL: https://issues.apache.org/jira/browse/HDFS-2757
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 1.1.0
>
>         Attachments: HDFS-2757-branch-1.patch, HDFS-2757-branch-1-v2.patch, HDFS-2757-trunk.patch,
hdfs-2757.txt
>
>
> When testing the tail'ing of a local file with the read short circuit on, I get:
> {noformat}
> 2012-01-06 00:17:31,598 WARN org.apache.hadoop.hdfs.DFSClient: BlockReaderLocal requested
with incorrect offset:  Offset 0 and length 8230400 don't match block blk_-2842916025951313698_454072
( blockLen 124 )
> 2012-01-06 00:17:31,598 WARN org.apache.hadoop.hdfs.DFSClient: BlockReaderLocal: Removing
blk_-2842916025951313698_454072 from cache because local file /export4/jdcryans/dfs/data/blocksBeingWritten/blk_-2842916025951313698
could not be opened.
> 2012-01-06 00:17:31,599 INFO org.apache.hadoop.hdfs.DFSClient: Failed to read block blk_-2842916025951313698_454072
on local machine java.io.IOException:  Offset 0 and length 8230400 don't match block blk_-2842916025951313698_454072
( blockLen 124 )
> 2012-01-06 00:17:31,599 INFO org.apache.hadoop.hdfs.DFSClient: Try reading via the datanode
on /10.4.13.38:51010
> java.io.EOFException: hdfs://sv4r11s38:9100/hbase-1/.logs/sv4r13s38,62023,1325808100311/sv4r13s38%2C62023%2C1325808100311.1325808100818,
entryStart=7190409, pos=8230400, end=8230400, edit=5
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message