hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8337) Investigate why disabling hadoop short circuit read is required to make recovery tests pass consistently under hadoop2
Date Fri, 19 Apr 2013 23:13:15 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636997#comment-13636997
] 

Jonathan Hsieh commented on HBASE-8337:
---------------------------------------

Chatted with [~cmccabe].  There are essentially three options.  1) disable scr. 2) enable
old-style insecure scr. 3) enable new-style (hadoop3 / CDH4.0) scr.

* check for native code enabled and classloader knows about BlockReaderLocalLegacy => new
style SCR
* else => old style SCR

* configuring new-style SCR
** dfs.client.read.shortcircuit = true
    ** dfs.client.read.shortcircuit.skip.checksum = false
    ** call DomainSocket.disableBindPathValidation();
    ** new TemporarySocketDirectory()
       *** dfs.domain.socket.path = tempSockDir.getPath();
            
* configuring old-style (insecure) SCR
    ** dfs.client.read.shortcircuit = true
    ** dfs.client.read.shortcircuit.skip.checksum = false
    ** dfs.block.local-path-access.user = <comma-separated list of usernames that will
be used to access>
    ** dfs.datanode.data.dir.perm = 755
                
> Investigate why disabling hadoop short circuit read is required to make recovery tests
pass consistently under hadoop2
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8337
>                 URL: https://issues.apache.org/jira/browse/HBASE-8337
>             Project: HBase
>          Issue Type: Sub-task
>          Components: hadoop2, test
>    Affects Versions: 0.98.0, 0.95.1
>            Reporter: Jonathan Hsieh
>            Priority: Critical
>             Fix For: 0.95.1
>
>
> HBASE-7636 makes some TestDistributedLogSplitting pass consistently by disabling hdfs
short circuit reads.  
> HBASE-8349 makes datanode node death recovery pass consistently by disabling hdfs short
circuit reads.
> This will likely require configuration modifications to fix and may have different fixes
for hadoop1, hadoop2 (HDFS-2246), and hadoop3 (HDFS-347)...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message