flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1579) Create a Flink History Server
Date Tue, 14 Mar 2017 14:17:47 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924286#comment-15924286
] 

ASF GitHub Bot commented on FLINK-1579:
---------------------------------------

Github user uce commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3460#discussion_r105905806
  
    --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala
---
    @@ -183,6 +198,45 @@ class MemoryArchivist(private val max_entries: Int)
         }
       }
     
    +  private def archiveJsonFiles(graph: ArchivedExecutionGraph) {
    +    future {
    +      try {
    +        val rootPath = new Path(flinkConfiguration.getString(
    --- End diff --
    
    Since this has no default value, we need to check whether the path is null. You should
check this before submitting the future to safe work when no directory is configured.
    
    I get the following Exception in the logs:
    
    ```
    2017-03-14 14:24:41,953 ERROR org.apache.flink.runtime.jobmanager.MemoryArchivist    
      - Failed to archive job.
    java.lang.IllegalArgumentException: Can not create a Path from a null string
    	at org.apache.flink.core.fs.Path.checkAndTrimPathArg(Path.java:159)
    	at org.apache.flink.core.fs.Path.<init>(Path.java:176)
    	at org.apache.flink.runtime.jobmanager.MemoryArchivist$$anonfun$org$apache$flink$runtime$jobmanager$MemoryArchivist$$archiveJsonFiles$1.apply$mcV$sp(MemoryArchivist.scala:204)
    	at org.apache.flink.runtime.jobmanager.MemoryArchivist$$anonfun$org$apache$flink$runtime$jobmanager$MemoryArchivist$$archiveJsonFiles$1.apply(MemoryArchivist.scala:203)
    	at org.apache.flink.runtime.jobmanager.MemoryArchivist$$anonfun$org$apache$flink$runtime$jobmanager$MemoryArchivist$$archiveJsonFiles$1.apply(MemoryArchivist.scala:203)
    	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
    	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
    	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
    ```


> Create a Flink History Server
> -----------------------------
>
>                 Key: FLINK-1579
>                 URL: https://issues.apache.org/jira/browse/FLINK-1579
>             Project: Flink
>          Issue Type: New Feature
>          Components: Distributed Coordination
>    Affects Versions: 0.9
>            Reporter: Robert Metzger
>            Assignee: Chesnay Schepler
>
> Right now its not possible to analyze the job results for jobs that ran on YARN, because
we'll loose the information once the JobManager has stopped.
> Therefore, I propose to implement a "Flink History Server" which serves  the results
from these jobs.
> I haven't started thinking about the implementation, but I suspect it involves some JSON
files stored in HDFS :)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message