hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6433) Add AsyncDiskService that is used in both hdfs and mapreduce
Date Fri, 11 Dec 2009 06:38:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789142#action_12789142

Zheng Shao commented on HADOOP-6433:

bq.  Can you convert the unit test to JUnit 4 style? (ie using annotations instead of extending
Added "@Test".

bq. Can you add a test case for a volume that is not in vols?

bq. Rather than sleeping in the test case, can you call shutdown and then awaitTermination()
with a longer timeout? This should ensure that the test is allowed to run longer if the machine
is super slow, but doesn't waste time if not necessary. This would also make you add an awaitTermination()
call, which I think is a good thing to expose.
Yes. Done.

bq. What's with the changes to FsShell? Was this accidental?
Yes. Removed.

bq. Missing license header on AsyncDIskService.java

bq. The comment still says "In the future, we should put it in common". Since this is a common
jira, please remove that

bq. IMO the thread counts per volume should be configurable. Some systems can handle plenty
of concurrent metadata ops while others, not so much. I don't feel strongly about this, though
- happy to leave it to a later JIRA if people want this.
If the system can handle a lot of concurrent ops, that usually means the system is fast, so
that we should be able to finish the ops pretty quickly. As a result I think there is not
much need to change this option in reality. Let's postpone it until people ask for it.

bq. Should the threads be made daemon threads? I'm not sure what the expected behavior should
be when the server gets SIGINTed, but I think if they're not daemon threads it will block
server shutdown.
Added "shutdownNow()" so that people can force shutdown.

> Add AsyncDiskService that is used in both hdfs and mapreduce
> ------------------------------------------------------------
>                 Key: HADOOP-6433
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6433
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: Zheng Shao
>            Assignee: Zheng Shao
>         Attachments: HADOOP-6433.1.patch, HADOOP-6433.2.patch
> Both MAPREDUCE-1213 and HDFS-611 are using a class called AsyncDiskService.
> The idea is to create a thread pool per disk volume, and use that for scheduling async
disk operations.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message