hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhihai xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
Date Fri, 25 Sep 2015 17:35:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908372#comment-14908372

zhihai xu commented on HADOOP-8437:

thanks [~brahmareddy] for the patch!  If the size of {{dirs}} is zero, it will cause the following
java.lang.IllegalArgumentException: n must be positive
at java.util.Random.nextInt(Random.java:250)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:243)
same exception found at HADOOP-6766 and MAPREDUCE-635.
Can we avoid IllegalArgumentException by checking {{dirs.size()}} non-zero before calling
{{dirIndexRandomizer.nextInt}} with the following simpler change?
if (dirs.size() > 0) {
  // randomize the first disk picked in the round-robin selection 
  dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
then it will trigger {{DiskErrorException}} later due to zero length {{localDirs}} if calling
from {{getLocalPathForWrite}} or {{getLocalPathToRead}} .

> getLocalPathForWrite is not throwing any expection for invalid paths
> --------------------------------------------------------------------
>                 Key: HADOOP-8437
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8437
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>              Labels: BB2015-05-TBR
>         Attachments: HADOOP-8437-003.patch, HADOOP-8437.patch, HADOOP-8437_1.patch, HADOOP-8437_2.patch
> call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf );
> Here it will not thrown any exception but earlier version it used throw.

This message was sent by Atlassian JIRA

View raw message