tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saikat (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (TEZ-2613) Fetcher(unordered) using List to store InputAttemptIdentifier can lead to some inefficiency during remove() operation
Date Mon, 27 Jul 2015 15:40:05 GMT

    [ https://issues.apache.org/jira/browse/TEZ-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642844#comment-14642844
] 

Saikat edited comment on TEZ-2613 at 7/27/15 3:39 PM:
------------------------------------------------------

Hi [~rajesh.balamohan]  thanks for your review comments.
Changed the iterator in getNextRemainingAttempt

For the setupConnection, I had a doubt. Shouldn't we use the remaining identifier while setting
up connection.(because setup connection may also be called again in the retry scenario line
528). 

My suggestion would be to pass the remainingSrcAttemptsList as an Iterable<InputAttemptIdentifier>
to setupConnection.That we can avoid creating a new linkedlist.

e.g.
function prototype for setupConnection-
private HostFetchResult setupConnection(Iterable<InputAttemptIdentifier> attempts);

//called as 
HostFetchResult connectionsWithRetryResult =
        setupConnection(srcAttemptsRemaining.values());

Patchset 4 shows the suggested change.


was (Author: saikatr):
Hi [~rajesh.balamohan]  thanks for your review comments.
Changed the iterator in getNextRemainingAttempt

For the setupConnection, I had a doubt. Shouldn't we use the remaining identifier while setting
up connection.(because setup connection may also be called again in the retry scenario line
528). Once I get clarification on this behaviour, will submit a change accordingly.

My suggestion would be to pass the remainingSrcAttemptsList as an Iterable<InputAttemptIdentifier>
to setupConnection.That we can avoid creating a new linkedlist.

e.g.
function prototype for setupConnection-
private HostFetchResult setupConnection(Iterable<InputAttemptIdentifier> attempts);

//called as 
HostFetchResult connectionsWithRetryResult =
        setupConnection(srcAttemptsRemaining.values());

Patchset 4 shows the suggested change.

> Fetcher(unordered) using List to store InputAttemptIdentifier can lead to some inefficiency
during remove() operation
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: TEZ-2613
>                 URL: https://issues.apache.org/jira/browse/TEZ-2613
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Saikat
>            Assignee: Saikat
>         Attachments: TEZ-2613.1.patch, TEZ-2613.2.patch, TEZ-2613.3.patch, TEZ-2613.4.patch,
TEZ-2613.patch
>
>
> remove() operation on the remaining list can be inefficient. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message