flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6295) use LoadingCache instead of WeakHashMap to lower latency
Date Thu, 13 Apr 2017 04:07:42 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967091#comment-15967091
] 

ASF GitHub Bot commented on FLINK-6295:
---------------------------------------

Github user wenlong88 commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3709#discussion_r111308326
  
    --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/ExecutionGraphHolder.java
---
    @@ -48,7 +52,41 @@
     
     	private final FiniteDuration timeout;
     
    -	private final WeakHashMap<JobID, AccessExecutionGraph> cache = new WeakHashMap<>();
    +	private AtomicReference<ActorGateway> jobManagerRef = new AtomicReference<>(null);
    +
    +	private final LoadingCache<JobID, AccessExecutionGraph> cache =
    +		CacheBuilder.newBuilder()
    +			.maximumSize(1000)
    +			.expireAfterWrite(30, TimeUnit.SECONDS)
    +			.build(new CacheLoader<JobID, AccessExecutionGraph>() {
    +				@Override
    +				public AccessExecutionGraph load(JobID jobID) throws Exception {
    +					try {
    +						if (jobManagerRef.get() != null) {
    +							Future<Object> future = jobManagerRef.get().ask(new JobManagerMessages.RequestJob(jobID),
timeout);
    +							Object result = Await.result(future, timeout);
    +
    +							if (result instanceof JobManagerMessages.JobNotFound) {
    +								return null;
    --- End diff --
    
    CacheLoader do not support return null.


> use LoadingCache instead of WeakHashMap to lower latency
> --------------------------------------------------------
>
>                 Key: FLINK-6295
>                 URL: https://issues.apache.org/jira/browse/FLINK-6295
>             Project: Flink
>          Issue Type: Bug
>          Components: Webfrontend
>            Reporter: Tao Wang
>            Assignee: Tao Wang
>
> Now in ExecutionGraphHolder, which is used in many handlers, we use a WeakHashMap to
cache ExecutionGraph(s), which is only sensitive to garbage collection.
> The latency is too high when JVM do GC rarely, which will make status of jobs or its
tasks unmatched with the real ones.
> LoadingCache is a common used cache implementation from guava lib, we can use its time
based eviction to lower latency of status update.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message