hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11846) Ozone: Potential http connection leaks in ozone clients
Date Wed, 24 May 2017 01:08:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022171#comment-16022171
] 

Weiwei Yang commented on HDFS-11846:
------------------------------------

Thanks [~xyao] for reviewing this. I uploaded v2 patch which has following changes

bq. OzoneClientUtils.java: Do we need a configure for HTTP Connection Timeout?

Added another property {{OZONE_CLIENT_CONNECTION_TIMEOUT_MS}} for the connection timeout.

bq. OzoneBucket.java: we should ensure the outPutStream is closed properly in the final section
as well.

I have fixed this line, and I have reviewed all streams in these classes and make sure they
are closed in final statement as well.

bq. OzoneClient.java:  unnecessary change.

Fixed

bq. OzoneClient.java: httppost-> httpPost?

I have fixed all such names in {{OzoneClient}}, {{OzoneVolume}} and {{OzoneBucket}} to make
sure they all be consistent.

bq. OzoneClient.java: I think we should use PoolingHttpClientConnectionManager instead of
creating a new connection for each request.

I tried to use PoolingHttpClientConnectionManager but I found it doesn't work in current code
base. The problem was on the netty server side. It doesn't serve a reused http connection
well, I think we need a new JIRA to fix that.

Thank you.

> Ozone: Potential http connection leaks in ozone clients
> -------------------------------------------------------
>
>                 Key: HDFS-11846
>                 URL: https://issues.apache.org/jira/browse/HDFS-11846
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: HDFS-11846-HDFS-7240.001.patch, HDFS-11846-HDFS-7240.002.patch
>
>
> There are several problems
> # Http clients in {{OzoneVolume}}, {{OzoneBucket}} and {{OzoneClient}} are created per
request, per [Reuse of HttpClient instance|http://hc.apache.org/httpclient-3.x/performance.html#Reuse_of_HttpClient_instance]
doc, proposed to reuse the http client instance to reduce the over head.
> # Some resources in these classes were not properly cleaned up. E.g the http connection,
HttpGet/HttpPost requests.
>  
> This jira's purpose is to fix these issues and investigate how we can improve the client.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message