hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-914) Refactor DFSOutputStream and DFSInputStream out of DFSClient
Date Mon, 08 Feb 2010 22:43:28 GMT

    [ https://issues.apache.org/jira/browse/HDFS-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831172#action_12831172

Konstantin Shvachko commented on HDFS-914:

"already locked" is the consequence of the initial error in {{TestDatanodeBlockScanner.testDatanodeBlockScanner()}},
which is
java.io.FileNotFoundException: http://localhost:53291/blockScannerReport?listblocks
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1288)
	at org.apache.hadoop.hdfs.DFSTestUtil.urlGet(DFSTestUtil.java:286)
	at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.waitForVerification(TestDatanodeBlockScanner.java:70)
	at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.testDatanodeBlockScanner(TestDatanodeBlockScanner.java:129)
Then the test fails to exit all threads, and the directory remains locked for subsequent tests.
I think we should file a separate issue for fixing {{TestDatanodeBlockScanner}}.

> Refactor DFSOutputStream and DFSInputStream out of DFSClient
> ------------------------------------------------------------
>                 Key: HDFS-914
>                 URL: https://issues.apache.org/jira/browse/HDFS-914
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs client
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-914.txt, hdfs-914.txt, hdfs-914.txt
> I'd like to propose refactoring DFSClient to extract DFSOutputStream and DFSInputStream
into a new org.apache.hadoop.hdfs.client package. DFSClient has become unmanageably large,
containing 8 inner classes.and approaching 4kloc. Factoring out the non-static inner classes
will also make them easier to test in isolation.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message