Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 925E2200B78 for ; Thu, 18 Aug 2016 17:39:25 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9104F160AAE; Thu, 18 Aug 2016 15:39:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D8021160A86 for ; Thu, 18 Aug 2016 17:39:24 +0200 (CEST) Received: (qmail 69545 invoked by uid 500); 18 Aug 2016 15:39:22 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 68988 invoked by uid 99); 18 Aug 2016 15:39:22 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Aug 2016 15:39:22 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 240262C02B3 for ; Thu, 18 Aug 2016 15:39:22 +0000 (UTC) Date: Thu, 18 Aug 2016 15:39:22 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (FLINK-1984) Integrate Flink with Apache Mesos MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 18 Aug 2016 15:39:25 -0000 [ https://issues.apache.org/jira/browse/FLINK-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426661#comment-15426661 ] ASF GitHub Bot commented on FLINK-1984: --------------------------------------- Github user EronWright commented on a diff in the pull request: https://github.com/apache/flink/pull/2315#discussion_r75332580 --- Diff: flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/TaskMonitor.scala --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.mesos.scheduler + +import grizzled.slf4j.Logger + +import akka.actor.{Actor, FSM, Props} +import org.apache.flink.configuration.Configuration +import org.apache.flink.mesos.scheduler.ReconciliationCoordinator.Reconcile +import org.apache.flink.mesos.scheduler.TaskMonitor._ +import org.apache.flink.mesos.scheduler.messages.{Connected, Disconnected, StatusUpdate} +import org.apache.mesos.Protos.TaskState._ +import org.apache.mesos.{SchedulerDriver, Protos} + +import scala.PartialFunction.empty +import scala.concurrent.duration._ + +/** + * Monitors a Mesos task throughout its lifecycle. + * + * Models a task with a state machine reflecting the perceived state of the task in Mesos. The state + * is primarily updated when task status information arrives from Mesos. + * + * The associated state data primarily tracks the task's goal (intended) state, as persisted by the scheduler. + * Keep in mind that goal state is persisted before actions are taken. The goal state strictly transitions + * thru New->Launched->Released. + * + * Unlike most exchanges with Mesos, task status is delivered at-least-once, so status handling should be idempotent. + */ +class TaskMonitor( + flinkConfig: Configuration, + schedulerDriver: SchedulerDriver, + goalState: TaskGoalState) extends Actor with FSM[TaskMonitorState,StateData] { + + val LOG = Logger(getClass) + + startWith(Suspended, StateData(goalState)) + + // ------------------------------------------------------------------------ + // Suspended State + // ------------------------------------------------------------------------ + + when(Suspended) { + case Event(update: TaskGoalStateUpdated, _) => + stay() using StateData(update.state) + case Event(msg: StatusUpdate, _) => + stay() + case Event(msg: Connected, StateData(goal: New)) => + goto(New) + case Event(msg: Connected, StateData(goal: Launched)) => + goto(Reconciling) + case Event(msg: Connected, StateData(goal: Released)) => + goto(Killing) + } + + // ------------------------------------------------------------------------ + // New State + // ------------------------------------------------------------------------ + + when(New) { + case Event(TaskGoalStateUpdated(goal: Launched), _) => + goto(Staging) using StateData(goal) + } + + // ------------------------------------------------------------------------ + // Reconciliation State + // ------------------------------------------------------------------------ + + onTransition { + case _ -> Reconciling => + nextStateData.goal match { + case goal: Launched => + val taskStatus = Protos.TaskStatus.newBuilder() + .setTaskId(goal.taskID).setSlaveId(goal.slaveID).setState(TASK_STAGING).build() + context.parent ! Reconcile(Seq(taskStatus)) --- End diff -- I agree that the alternative to using the implicit parent reference is to always use an explicit reference. It can be unit tested either way. I actually considered the latter but couldn't think of a nice name for the explicit reference. > Integrate Flink with Apache Mesos > --------------------------------- > > Key: FLINK-1984 > URL: https://issues.apache.org/jira/browse/FLINK-1984 > Project: Flink > Issue Type: New Feature > Components: Cluster Management > Reporter: Robert Metzger > Assignee: Eron Wright > Priority: Minor > Attachments: 251.patch > > > There are some users asking for an integration of Flink into Mesos. > -There also is a pending pull request for adding Mesos support for Flink-: https://github.com/apache/flink/pull/251 > Update (May '16): a new effort is now underway, building on the recent ResourceManager work. > Design document: ([google doc|https://docs.google.com/document/d/1WItafBmGbjlaBbP8Of5PAFOH9GUJQxf5S4hjEuPchuU/edit?usp=sharing]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)