hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nandakumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12178) Ozone: OzoneClient: Handling SCM container creationFlag at client side
Date Mon, 24 Jul 2017 19:09:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16098989#comment-16098989
] 

Nandakumar commented on HDFS-12178:
-----------------------------------

Thanks [~vagarychen] for the review.
> Curious though, is there a case where multiple threads will try to create the same container?
Since we are not checking {{keyInfo.getShouldCreateContainer()}} for container creation, all
{{createKey}} call will try to create container (if not in local cache {{containersCreated}}).
And tools like Corona will start multiple threads to put data into ozone, in such scenarios
multiple threads will try to create same container.
> createContainer can fail, but it already gets added to containersCreated set to this
point
It is assumed that {{ContainerProtocolCalls.createContainer(xceiverClient, requestId)}} call
will fail most of the time, since we will be trying to create an already existing container.
If we switch the order of those two statements, we will never be adding containerName to the
cache since there will always be exception thrown during createContainer call (in case of
existing container), and all the {{createKey}} call will always try to create container which
is same as not having {{containersCreated}} cache.
>or simply move containersCreated.add() to after the try-catch?
This will work, but it doesn't make much of a difference, whether we have it as first statement
in try or move it totally outside of try-catch block. Both will behave the same way.

> Ozone: OzoneClient: Handling SCM container creationFlag at client side
> ----------------------------------------------------------------------
>
>                 Key: HDFS-12178
>                 URL: https://issues.apache.org/jira/browse/HDFS-12178
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Nandakumar
>            Assignee: Nandakumar
>         Attachments: HDFS-12178-HDFS-7240.000.patch, HDFS-12178-HDFS-7240.001.patch
>
>
> SCM BlockManager provisions a pool of containers upon block creation request, only one
container is returned with creationFlag to the client. The other containers provisioned in
the same batch will not have this flag. This jira is to handle that scenario at client side,
until  HDFS-11888 is fixed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message