hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11873) Ozone: Object store handler cannot serve multiple requests from single http client
Date Sat, 16 Sep 2017 03:55:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168801#comment-16168801
] 

Weiwei Yang commented on HDFS-11873:
------------------------------------

Hi [~xyao]

Thanks for fixing this, this should have fixed issue. I am not familiar with netty stuff,
it might be good to have someone else to review this patch. One thing small, when I run the
test case, it seems to create following dirs/files under {{~/hadoop/hadoop-hdfs-project/hadoop-hdfs}}

{noformat}
ls -R true/
_objects/    metadata.db/ user.db/

true//_objects:

true//metadata.db:
000003.log       CURRENT          IDENTITY         LOCK             LOG              MANIFEST-000001
 OPTIONS-000005

true//user.db:
000003.log       CURRENT          IDENTITY         LOCK             LOG              MANIFEST-000001
 OPTIONS-000005
{noformat}

any idea why?

> Ozone: Object store handler cannot serve multiple requests from single http client
> ----------------------------------------------------------------------------------
>
>                 Key: HDFS-11873
>                 URL: https://issues.apache.org/jira/browse/HDFS-11873
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Xiaoyu Yao
>            Priority: Critical
>              Labels: ozoneMerge
>         Attachments: HDFS-11873-HDFS-7240.001.patch, HDFS-11873-HDFS-7240.002.patch,
HDFS-11873-HDFS-7240.testcase.patch
>
>
> This issue was found when I worked on HDFS-11846. Instead of creating a new http client
instance per request, I tried to reuse {{CloseableHttpClient}} in {{OzoneClient}} class in
a {{PoolingHttpClientConnectionManager}}. However, every second request from the http client
hangs, which could not get dispatched to {{ObjectStoreJerseyContainer}}. There seems to be
something wrong in the netty pipeline, this jira aims to 1) fix the problem in the server
side 2) use the pool for client http clients to reduce the resource overhead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message