hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Jacobs (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3446) HostsFileReader silently ignores bad includes/excludes
Date Tue, 05 Jun 2012 19:05:23 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthew Jacobs updated HDFS-3446:
---------------------------------

    Status: Patch Available  (was: Open)

Overview:
This patch causes the HostsFileReader to throw an exception if the include/excludes files
don't exist when they are used.
 * Prevents overwriting existing include/exclude lists with empty sets.
 * Avoids failing silently

Background:
The HostsFileReader silently fails if the includes or excludes files do not exist or are not
readable. The current behavior is to overwrite the existing set of hosts or excluded hosts,
regardless of whether or not the includes/excludes file exists. This behavior was introduced
in HADOOP-5643 to support updating the job tracker's node lists. The HostsFileReader is no
longer used by the job tracker. If this behavior was intentional, it no longer seems necessary
for any reason.

A better behavior is to prevent a non-existent/unreadable file from being set or used. This
patch ensures that the HostsFileReader will throw an exception in these cases. The HostsFileReader
is still used by NodeListManager as well as the DatanodeManager, and in both cases, throwing
an exception when the include/exclude files aren't found/readable is desirable.

Testing:
There are a few cases to test, all of which now have unit tests in TestHostsFileReader:
(1) Creating a new HostsFileReader with a non-existent/unreadable file
(2) Refreshing a HostsFileReader with an include/exclude file that no longer exists or is
no longer readable
(3) Setting the include/exclude file to a file that doesn't exist or isn't readable

In addition, there were some unrelated unit tests that previously relied on the faulty behavior
of HostsFileReader such that they didn't have to set valid includes or excludes files. Those
tests have been updated.
                
> HostsFileReader silently ignores bad includes/excludes
> ------------------------------------------------------
>
>                 Key: HDFS-3446
>                 URL: https://issues.apache.org/jira/browse/HDFS-3446
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 1.0.0
>            Reporter: Matthew Jacobs
>            Assignee: Matthew Jacobs
>
> The HostsFileReader silently fails if the includes or excludes files do
> not exist or are not readable. The current behavior is to overwrite the
> existing set of hosts or excluded hosts, regardless of whether or not
> the includes/excludes file exists.
> This behavior was introduced in HADOOP-5643 to support updating the job
> tracker's node lists. The HostsFileReader is no longer used by the job
> tracker. If this behavior was intentional, it no longer seems necessary
> for any reason. The HostsFileReader is still used by NodeListManager as
> well as the DatanodeManager, and in both cases, throwing an exception
> when the include/exclude files aren't found/readable is desirable.
> We should validate the given includes and excludes files before using
> them.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message