hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8337) Investigate why disabling hadoop short circuit read is required to make recovery tests pass consistently under hadoop2
Date Wed, 24 Apr 2013 21:27:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640976#comment-13640976
] 

Colin Patrick McCabe commented on HBASE-8337:
---------------------------------------------

bq. Looks like this has allowed us to get away with things we shouldn't. Tested using the
same User for master and all regionservers in the minicluster, with 0.94 branch and the default
Hadoop 1. TestMasterZKSessionRecovery OOMEs after surefire tries to parse a 180 MB logfile
full of IOExceptions. As soon as one regionserver aborts, its filesystem is cached and/or
closed by user, the master file system's DFS client is closed, and all hell breaks loose.

You can use {{Filesystem#newInstance}} to prevent this problem.
                
> Investigate why disabling hadoop short circuit read is required to make recovery tests
pass consistently under hadoop2
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8337
>                 URL: https://issues.apache.org/jira/browse/HBASE-8337
>             Project: HBase
>          Issue Type: Sub-task
>          Components: hadoop2, test
>    Affects Versions: 0.98.0, 0.95.1
>            Reporter: Jonathan Hsieh
>            Priority: Critical
>             Fix For: 0.95.1
>
>
> HBASE-7636 makes some TestDistributedLogSplitting pass consistently by disabling hdfs
short circuit reads.  
> HBASE-8349 makes datanode node death recovery pass consistently by disabling hdfs short
circuit reads.
> This will likely require configuration modifications to fix and may have different fixes
for hadoop1, hadoop2 (HDFS-2246), and hadoop3 (HDFS-347)...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message