hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-15415) copyBytes hangs when the configuration file is corrupted
Date Fri, 27 Apr 2018 10:09:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HADOOP-15415:
------------------------------------
    Affects Version/s: 3.1.0

> copyBytes hangs when the configuration file is corrupted
> --------------------------------------------------------
>
>                 Key: HADOOP-15415
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15415
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common
>    Affects Versions: 0.23.0, 3.1.0
>            Reporter: John Doe
>            Priority: Major
>
> The third parameter,  buffSize, is read from the configuration files or user-specified.
> When the configuration file is corrupted or the user configures with a wrong value, i.e.,
0, the bytesRead will always be 0, making the while loop's condition always true, hanging
IOUtils.copyBytes() endlessly.
> Here is the snippet of the code. There are four copyBytes in the following code. The
3rd and 4th copyBytes calls the 1st one. The 1st one calls the 2nd one. Hang happens in the
while loop of the second copyBytes function.
>  
> {code:java}
> //1st copyBytes
>   public static void copyBytes(InputStream in, OutputStream out, int buffSize, boolean
close) 
>     throws IOException {
>     try {
>       copyBytes(in, out, buffSize);
>       if(close) {
>         out.close();
>         out = null;
>         in.close();
>         in = null;
>       }
>     } finally {
>       if(close) {
>         closeStream(out);
>         closeStream(in);
>       }
>     }
>   }
>   
> //2nd copyBytes
>   public static void copyBytes(InputStream in, OutputStream out, int buffSize) 
>     throws IOException {
>     PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;
>     byte buf[] = new byte[buffSize];
>     int bytesRead = in.read(buf);
>     while (bytesRead >= 0) {
>       out.write(buf, 0, bytesRead);
>       if ((ps != null) && ps.checkError()) {
>         throw new IOException("Unable to write to output stream.");
>       }
>       bytesRead = in.read(buf);
>     }
>   }
> //3rd copyBytes
>   public static void copyBytes(InputStream in, OutputStream out, Configuration conf)
>     throws IOException {
>     copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), true);
>   }
>   
> //4th copyBytes
>   public static void copyBytes(InputStream in, OutputStream out, Configuration conf,
boolean close)
>     throws IOException {
>     copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096),  close);
>   }
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message