hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7017) Implement OutputStream for libhdfs3
Date Fri, 14 Nov 2014 07:39:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211985#comment-14211985

Colin Patrick McCabe commented on HDFS-7017:

bq. I agree that this indirection makes the code hard to follow, Colin, would you please recommend
a better way to do such unit test?

It's hard to give a single answer to this because there's a lot of different strategies for
implementing unit tests.  In general, if you look at our Java code, we use many different
strategies for unit testing.

One strategy that can be pretty effective is just changing the configuration.  For example,
if you want to test lease timeouts, you could set a configuration that causes the lease to
time out almost instantly, or even after 0 time.  Another strategy is having a "fault injector"
object like {{DFSClientFaultInjector}} or {{CheckpointFaultInjector}}.  "Fault injectors"
can be easier to understand than making every method virtual.  So you could inject a fault
where perhaps the heartbeat packets are not actually sent, and the lease timeout path gets
tested (due to the lack of NN acknowledgement).

Another strategy is to write code that's modular so you don't have to mock things.  For example,
you should be able to test URL parsing or configuration parsing without actually creating
a FileSystem object, so there's no need for a mock in this case.  This isn't always possible,
of course.

Sometimes having virtual interfaces and mocks is the right solution, but it's definitely not
always the right solution.  It's great that you are thinking of unit tests though...

> Implement OutputStream for libhdfs3
> -----------------------------------
>                 Key: HDFS-7017
>                 URL: https://issues.apache.org/jira/browse/HDFS-7017
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Zhanwei Wang
>            Assignee: Zhanwei Wang
>         Attachments: HDFS-7017-pnative.002.patch, HDFS-7017-pnative.003.patch, HDFS-7017-pnative.004.patch,
> Implement pipeline and OutputStream C++ interface

This message was sent by Atlassian JIRA

View raw message