hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10429) DataStreamer interrupted warning always appears when using CLI upload file
Date Fri, 07 Oct 2016 20:52:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556220#comment-15556220
] 

Wei-Chiu Chuang commented on HDFS-10429:
----------------------------------------

I think that closeThreads(false) should be called before try...completeFile to allow response
processors to close gracefully while the client asks NN to complete the file. It is possible
that the threads are not closed when the forceful close() is called, in which case reducing
the log level from warn to debug can eliminate user frustration.

> DataStreamer interrupted warning  always appears when using CLI upload file
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-10429
>                 URL: https://issues.apache.org/jira/browse/HDFS-10429
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.3
>            Reporter: Zhiyuan Yang
>            Assignee: Zhiyuan Yang
>            Priority: Minor
>         Attachments: HDFS-10429.1.patch, HDFS-10429.2.patch
>
>
> Every time I use 'hdfs dfs -put' upload file, this warning is printed:
> {code:java}
> 16/05/18 20:57:56 WARN hdfs.DataStreamer: Caught exception
> java.lang.InterruptedException
> 	at java.lang.Object.wait(Native Method)
> 	at java.lang.Thread.join(Thread.java:1245)
> 	at java.lang.Thread.join(Thread.java:1319)
> 	at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:871)
> 	at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:519)
> 	at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:696)
> {code}
> The reason is this: originally, DataStreamer::closeResponder always prints a warning
about InterruptedException; since HDFS-9812, DFSOutputStream::closeImpl  always forces threads
to close, which causes InterruptedException.
> A simple fix is to use debug level log instead of warning level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message