flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6434) There may be allocatedSlots leak in SlotPool
Date Thu, 02 Nov 2017 17:26:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236186#comment-16236186
] 

ASF GitHub Bot commented on FLINK-6434:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4937#discussion_r148601504
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/instance/SlotPool.java
---
    @@ -645,6 +668,21 @@ AllocatedSlots getAllocatedSlots() {
     		return allocatedSlots;
     	}
     
    +	@VisibleForTesting
    +	AvailableSlots getAvailableSlots() {
    +		return availableSlots;
    +	}
    +
    +	@VisibleForTesting
    +	int getNumOfWaitingForResourceRequests() {
    +		return waitingForResourceManager.size();
    +	}
    +
    +	@VisibleForTesting
    +	int getNumOfPendingRequests() {
    +		return pendingRequests.size();
    +	}
    --- End diff --
    
    I think we should not make internal state easily accessible because it will usually be
modified by the main thread. Also when checking a certain interleaving you might be falsely
entrapped that you can do something like
    ```
    slotPool.asyncAddPendingRequest()
    slotPool.getNumOfPendingRequests() // this returns +1 pending requests
    ```
    
    This is might work but sometimes it also does not work because the concurrent operation
has not been completed. I would like to make concurrent operations explicit by, for example,
returning a `CompletableFuture<Integer> getNumberOfPendingRequests` if at all.


> There may be allocatedSlots leak in SlotPool
> --------------------------------------------
>
>                 Key: FLINK-6434
>                 URL: https://issues.apache.org/jira/browse/FLINK-6434
>             Project: Flink
>          Issue Type: Bug
>          Components: Cluster Management
>            Reporter: shuai.xu
>            Assignee: shuai.xu
>            Priority: Major
>              Labels: flip-6
>
> If the call allocateSlot() from Execution to Slotpool timeout, the job will begin to
failover, but the pending request are still in SlotPool, if then a new slot register to SlotPool,
it may be fulfill the outdated pending request and be added to allocatedSlots, but it will
never be used and will never be recycled.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message