hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "binlijin (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-2680) In FileSinkOperator if the RecordWriter.write throw IOException, we should call RecordWriter's close method.
Date Mon, 26 Dec 2011 14:30:30 GMT
In FileSinkOperator if the RecordWriter.write throw IOException, we should call RecordWriter's
close method.
------------------------------------------------------------------------------------------------------------

                 Key: HIVE-2680
                 URL: https://issues.apache.org/jira/browse/HIVE-2680
             Project: Hive
          Issue Type: Improvement
            Reporter: binlijin
             Fix For: 0.9.0


Dynamic-partition Insert, if the partition that will be created is large, many files will
be created and if the input is large the DataNode's xceiverCount will easily exceeds the limit
of concurrent xcievers (default 1024), The RecordWriter.write(recordValue) will throw the
Exception "Could not read from stream". After an hour the lease timeout, there will many commitBlockSynchronization
requests and the namenode's load will be very high, so abortWriters should be called.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message