Return-Path: X-Original-To: apmail-tez-commits-archive@minotaur.apache.org Delivered-To: apmail-tez-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F1A1D18918 for ; Mon, 14 Sep 2015 21:18:26 +0000 (UTC) Received: (qmail 54406 invoked by uid 500); 14 Sep 2015 21:18:26 -0000 Delivered-To: apmail-tez-commits-archive@tez.apache.org Received: (qmail 54370 invoked by uid 500); 14 Sep 2015 21:18:26 -0000 Mailing-List: contact commits-help@tez.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tez.apache.org Delivered-To: mailing list commits@tez.apache.org Received: (qmail 54361 invoked by uid 99); 14 Sep 2015 21:18:26 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Sep 2015 21:18:26 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id AB467E07D6; Mon, 14 Sep 2015 21:18:26 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: hitesh@apache.org To: commits@tez.apache.org Message-Id: <61a8f6995e5c4f47b82f152f5574854e@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: tez git commit: TEZ-2876. Tez UI: Update vertex, task & attempt details page while in progress. (Sreenath Somarajapuram via hitesh) Date: Mon, 14 Sep 2015 21:18:26 +0000 (UTC) Repository: tez Updated Branches: refs/heads/branch-0.7 6b05e1965 -> d2d5f7eb5 TEZ-2876. Tez UI: Update vertex, task & attempt details page while in progress. (Sreenath Somarajapuram via hitesh) (cherry picked from commit c4817a76bda559eacd071e8f46ad4e351e8e8376) Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/tez/repo Commit: http://git-wip-us.apache.org/repos/asf/tez/commit/d2d5f7eb Tree: http://git-wip-us.apache.org/repos/asf/tez/tree/d2d5f7eb Diff: http://git-wip-us.apache.org/repos/asf/tez/diff/d2d5f7eb Branch: refs/heads/branch-0.7 Commit: d2d5f7eb5a35e91603a9af6cdecb64fba9c6a01d Parents: 6b05e19 Author: Hitesh Shah Authored: Mon Sep 14 14:17:06 2015 -0700 Committer: Hitesh Shah Committed: Mon Sep 14 14:18:19 2015 -0700 ---------------------------------------------------------------------- CHANGES.txt | 1 + tez-ui/src/main/webapp/app/scripts/app.js | 19 ++++++++ .../controllers/task_attempt_controller.js | 51 ++++++++++++++++++++ .../app/scripts/controllers/task_controller.js | 50 +++++++++++++++++++ .../scripts/controllers/vertex_controller.js | 47 ++++++++++++++++++ .../app/scripts/models/TimelineRestAdapter.js | 20 ++++++++ .../src/main/webapp/app/scripts/models/dag.js | 6 +++ .../webapp/app/scripts/models/task_attempt.js | 2 + tez-ui/src/main/webapp/app/scripts/router.js | 38 +++++++++++++++ tez-ui/src/main/webapp/app/styles/main.less | 10 ++++ .../main/webapp/app/templates/task/index.hbs | 4 ++ .../webapp/app/templates/taskAttempt/index.hbs | 4 ++ .../main/webapp/app/templates/vertex/index.hbs | 4 ++ 13 files changed, 256 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index 184552f..0e90920 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -6,6 +6,7 @@ Release 0.7.1: Unreleased INCOMPATIBLE CHANGES ALL CHANGES: + TEZ-2786. Tez UI: Update vertex, task & attempt details page while in progress. TEZ-2817. Tez UI: update in progress counter data for the dag vertices and tasks table TEZ-2813. Tez UI: add counter data for rest api calls to AM Web Services v2 TEZ-2660. Tez UI: need to show application page even if system metrics publish is disabled. http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/app.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/app.js b/tez-ui/src/main/webapp/app/scripts/app.js index 2ad5fdd..51d5ee8 100644 --- a/tez-ui/src/main/webapp/app/scripts/app.js +++ b/tez-ui/src/main/webapp/app/scripts/app.js @@ -242,6 +242,25 @@ App.ready = function () { } }); + App.AttemptInfoAdapter = App.AMInfoAdapter.extend({ + namespace: App.Configs.restNamespace.aminfoV2, + findQuery: function(store, type, query) { + var record = query.metadata; + delete query.metadata; + return this.ajax( + this.buildURL(Ember.String.pluralize(type.typeKey), + record.attemptID, Em.Object.create(record)), 'GET', { data: query}); + }, + buildURL: function(type, id, record) { + var url = this._super(type, undefined, record); + return url.replace('__app_id__', record.get('appID')) + .fmt(record.get('dagID'), record.get('taskID'), id, record.get('counters')); + }, + pathForType: function(typeName) { + return 'attemptsInfo?dagID=%@&taskID=%@&attemptID=%@&counters=%@'; + } + }); + }; $.ajaxPrefilter(function(options, originalOptions, jqXHR) { http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/controllers/task_attempt_controller.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/controllers/task_attempt_controller.js b/tez-ui/src/main/webapp/app/scripts/controllers/task_attempt_controller.js index ad8dccd..ebb8d32 100644 --- a/tez-ui/src/main/webapp/app/scripts/controllers/task_attempt_controller.js +++ b/tez-ui/src/main/webapp/app/scripts/controllers/task_attempt_controller.js @@ -23,6 +23,57 @@ App.TaskAttemptController = Em.ObjectController.extend(App.Helpers.DisplayHelper loading: true, + isActive: false, + + pollster: App.Helpers.EntityArrayPollster.create(), + + init: function () { + this._super(); + this.get('pollster').setProperties({ + entityType: 'attemptInfo', + mergeProperties: ['status', 'progress'], + store: this.get('store') + }); + }, + + setup: function () { + this.set('isActive', true); + }, + reset: function () { + this.set('isActive', false); + }, + + pollsterControl: function () { + if(this.get('task.vertex.dag.status') == 'RUNNING' && + this.get('task.vertex.dag.amWebServiceVersion') != '1' && + this.get('isActive')) { + this.get('pollster').start(); + } + else { + this.get('pollster').stop(); + } + }.observes('task.vertex.dag.status', 'task.vertex.dag.amWebServiceVersion', 'isActive'), + + pollsterOptionsObserver: function () { + var model = this.get('model'); + + this.get('pollster').setProperties( (model && model.get('status') != 'SUCCEEDED') ? { + targetRecords: [model], + options: { + appID: this.get('task.vertex.dag.applicationId'), + dagID: App.Helpers.misc.getIndexFromId(this.get('dagID')), + attemptID: '%@_%@_%@'.fmt( + App.Helpers.misc.getIndexFromId(this.get('vertexID')), + App.Helpers.misc.getIndexFromId(this.get('taskID')), + App.Helpers.misc.getIndexFromId(this.get('id')) + ) + } + } : { + targetRecords: [], + options: null + }); + }.observes('task.vertex.dag.applicationId', 'status', 'dagID', 'vertexID', 'id'), + loadAdditional: function(attempt) { var that = this; http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/controllers/task_controller.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/controllers/task_controller.js b/tez-ui/src/main/webapp/app/scripts/controllers/task_controller.js index b95620a..8893c87 100644 --- a/tez-ui/src/main/webapp/app/scripts/controllers/task_controller.js +++ b/tez-ui/src/main/webapp/app/scripts/controllers/task_controller.js @@ -23,6 +23,56 @@ App.TaskController = Em.ObjectController.extend(App.Helpers.DisplayHelper, App.M loading: true, + isActive: false, + + pollster: App.Helpers.EntityArrayPollster.create(), + + init: function () { + this._super(); + this.get('pollster').setProperties({ + entityType: 'taskInfo', + mergeProperties: ['status', 'progress'], + store: this.get('store') + }); + }, + + setup: function () { + this.set('isActive', true); + }, + reset: function () { + this.set('isActive', false); + }, + + pollsterControl: function () { + if(this.get('vertex.dag.status') == 'RUNNING' && + this.get('vertex.dag.amWebServiceVersion') != '1' && + this.get('isActive')) { + this.get('pollster').start(); + } + else { + this.get('pollster').stop(); + } + }.observes('vertex.dag.status', 'vertex.dag.amWebServiceVersion', 'isActive'), + + pollsterOptionsObserver: function () { + var model = this.get('model'); + + this.get('pollster').setProperties( (model && model.get('status') != 'SUCCEEDED') ? { + targetRecords: [model], + options: { + appID: this.get('vertex.dag.applicationId'), + dagID: App.Helpers.misc.getIndexFromId(this.get('dagID')), + taskID: '%@_%@'.fmt( + App.Helpers.misc.getIndexFromId(this.get('vertexID')), + App.Helpers.misc.getIndexFromId(this.get('id')) + ) + } + } : { + targetRecords: [], + options: null + }); + }.observes('vertex.dag.applicationId', 'status', 'dagID', 'vertexID', 'id'), + loadAdditional: function(task) { var that = this; var applicationId = App.Helpers.misc.getAppIdFromVertexId(task.get('vertexID')); http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/controllers/vertex_controller.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/controllers/vertex_controller.js b/tez-ui/src/main/webapp/app/scripts/controllers/vertex_controller.js index a4ba6e1..eeee1c9 100644 --- a/tez-ui/src/main/webapp/app/scripts/controllers/vertex_controller.js +++ b/tez-ui/src/main/webapp/app/scripts/controllers/vertex_controller.js @@ -22,12 +22,59 @@ App.VertexController = Em.ObjectController.extend(App.Helpers.DisplayHelper, App pageTitle: 'Vertex', loading: true, + isActive: false, + + pollster: App.Helpers.EntityArrayPollster.create(), + + init: function () { + this._super(); + this.get('pollster').setProperties({ + entityType: 'vertexInfo', + mergeProperties: ['status', 'progress'], + store: this.get('store') + }); + }, + + setup: function () { + this.set('isActive', true); + }, + reset: function () { + this.set('isActive', false); + }, + + pollsterControl: function () { + if(this.get('dag.status') == 'RUNNING' && + this.get('dag.amWebServiceVersion') != '1' && + this.get('isActive')) { + this.get('pollster').start(); + } + else { + this.get('pollster').stop(); + } + }.observes('dag.status', 'dag.amWebServiceVersion', 'isActive'), + + pollsterOptionsObserver: function () { + var model = this.get('model'); + + this.get('pollster').setProperties( (model && model.get('status') != 'SUCCEEDED') ? { + targetRecords: [model], + options: { + appID: this.get('applicationId'), + dagID: App.Helpers.misc.getIndexFromId(this.get('dagID')), + vertexID: App.Helpers.misc.getIndexFromId(this.get('id')) + } + } : { + targetRecords: [], + options: null + }); + }.observes('applicationId', 'status', 'dagID', 'id'), loadAdditional: function(vertex) { var loaders = [], that = this, applicationId = vertex.get('applicationId'); + // Irrespective of am version this will get the progress first. if (vertex.get('status') == 'RUNNING') { var vertexIdx = vertex.get('id').split('_').splice(-1).pop(); App.Helpers.misc.removeRecord(this.store, 'vertexProgress', vertexIdx); http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/models/TimelineRestAdapter.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/models/TimelineRestAdapter.js b/tez-ui/src/main/webapp/app/scripts/models/TimelineRestAdapter.js index fce7ab1..c295366 100644 --- a/tez-ui/src/main/webapp/app/scripts/models/TimelineRestAdapter.js +++ b/tez-ui/src/main/webapp/app/scripts/models/TimelineRestAdapter.js @@ -165,6 +165,12 @@ var timelineJsonToTaskAttemptMap = { diagnostics: 'otherinfo.diagnostics', counterGroups: 'otherinfo.counters.counterGroups', + progress: { + custom: function(source) { + return Em.get(source, 'otherinfo.status') == 'SUCCEEDED' ? 1 : null; + } + }, + inProgressLog: 'otherinfo.inProgressLogsURL', completedLog: 'otherinfo.completedLogsURL', @@ -225,6 +231,12 @@ var timelineJsonToVertexMap = { startTime: 'otherinfo.startTime', endTime: 'otherinfo.endTime', + progress: { + custom: function(source) { + return Em.get(source, 'otherinfo.status') == 'SUCCEEDED' ? 1 : null; + } + }, + status: 'otherinfo.status', hasFailedTaskAttempts: { custom: function(source) { @@ -559,4 +571,12 @@ App.TaskInfoSerializer = DS.RESTSerializer.extend({ taskInfo : rawPayload.tasks } } +}); + +App.AttemptInfoSerializer = DS.RESTSerializer.extend({ + normalizePayload: function(rawPayload) { + return { + attemptInfo : rawPayload.attempts + } + } }); \ No newline at end of file http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/models/dag.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/models/dag.js b/tez-ui/src/main/webapp/app/scripts/models/dag.js index 3e48cfe..7aa3915 100644 --- a/tez-ui/src/main/webapp/app/scripts/models/dag.js +++ b/tez-ui/src/main/webapp/app/scripts/models/dag.js @@ -441,6 +441,12 @@ App.TaskInfo = DS.Model.extend({ counters: DS.attr('object') }); +App.AttemptInfo = DS.Model.extend({ + progress: DS.attr('number'), + status: DS.attr('string'), + counters: DS.attr('object') +}); + App.KVDatum = DS.Model.extend({ key: DS.attr('string'), value: DS.attr('string'), http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/models/task_attempt.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/models/task_attempt.js b/tez-ui/src/main/webapp/app/scripts/models/task_attempt.js index d8a48e6..888408d 100644 --- a/tez-ui/src/main/webapp/app/scripts/models/task_attempt.js +++ b/tez-ui/src/main/webapp/app/scripts/models/task_attempt.js @@ -22,6 +22,8 @@ App.TaskAttempt = App.AbstractEntity.extend({ return id.indexOf(idPrefix) == 0 ? id.substr(idPrefix.length) : id; }.property('id'), + progress: DS.attr('number'), + // start time of the entity startTime: DS.attr('number'), http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/scripts/router.js ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/scripts/router.js b/tez-ui/src/main/webapp/app/scripts/router.js index a937387..88880e8 100644 --- a/tez-ui/src/main/webapp/app/scripts/router.js +++ b/tez-ui/src/main/webapp/app/scripts/router.js @@ -155,6 +155,9 @@ App.DagRoute = Em.Route.extend({ setupController: setupControllerFactory('Dag: %@ (%@)', 'name', 'id'), resetController: function() { this.controller.dostopAMInfoUpdateService(); + if(this.controller.reset) { + this.controller.reset(); + } } }); @@ -182,6 +185,11 @@ App.TaskRoute = Em.Route.extend({ afterModel: function(model) { return this.controllerFor('task').loadAdditional(model); }, + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, setupController: setupControllerFactory('Task: %@', 'id') }); @@ -195,6 +203,11 @@ App.VertexRoute = Em.Route.extend({ model: function(params) { return this.store.find('vertex', params.vertex_id); }, + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, afterModel: function(model) { return this.controllerFor('vertex').loadAdditional(model); }, @@ -244,11 +257,21 @@ App.TaskAttemptRoute = Em.Route.extend({ afterModel: function(model) { return this.controllerFor('task_attempt').loadAdditional(model); }, + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, setupController: setupControllerFactory('Task Attempt: %@', 'id') }); App.TaskAttemptsRoute = Em.Route.extend({ renderTemplate: renderTable, + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, setupController: setupControllerFactory('Task Attempt: %@', 'id') }); @@ -267,11 +290,21 @@ App.TezAppRoute = Em.Route.extend({ }); }); }, + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, setupController: setupControllerFactory('Application: %@', 'id') }); App.TezAppDagsRoute = Em.Route.extend({ renderTemplate: renderTable, + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, setupController: setupControllerFactory() }); @@ -300,6 +333,11 @@ App.DagCountersRoute = App.TaskCountersRoute = App.TaskAttemptCountersRoute = Em.Route.extend({ + resetController: function () { + if(this.controller.reset) { + this.controller.reset(); + } + }, renderTemplate: function() { this.render('common/counters'); } http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/styles/main.less ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/styles/main.less b/tez-ui/src/main/webapp/app/styles/main.less index 1df5753..7484519 100644 --- a/tez-ui/src/main/webapp/app/styles/main.less +++ b/tez-ui/src/main/webapp/app/styles/main.less @@ -358,6 +358,10 @@ body, html { table-layout: fixed; white-space: nowrap; + .progress { + margin-bottom: 0px; + } + tr { margin: 0px 0px 0px 15px; overflow: hidden; @@ -393,6 +397,12 @@ body, html { color: @success-color; } + &.schedule { + .fa-icon(history); + .fa-flip-horizontal; + color: @warning-color; + } + &.running { .fa-icon(spinner); .fa-spin; http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/templates/task/index.hbs ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/templates/task/index.hbs b/tez-ui/src/main/webapp/app/templates/task/index.hbs index f760797..30a0037 100644 --- a/tez-ui/src/main/webapp/app/templates/task/index.hbs +++ b/tez-ui/src/main/webapp/app/templates/task/index.hbs @@ -47,6 +47,10 @@ + Progress + {{bs-progress-animated progressDecimal=progress}} + + {{t 'common.time.start'}} {{formatUnixTimestamp startTime}} http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/templates/taskAttempt/index.hbs ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/templates/taskAttempt/index.hbs b/tez-ui/src/main/webapp/app/templates/taskAttempt/index.hbs index ee6452b..346c44f 100644 --- a/tez-ui/src/main/webapp/app/templates/taskAttempt/index.hbs +++ b/tez-ui/src/main/webapp/app/templates/taskAttempt/index.hbs @@ -55,6 +55,10 @@ + Progress + {{bs-progress-animated progressDecimal=progress}} + + {{t 'common.time.start'}} {{formatUnixTimestamp startTime}} http://git-wip-us.apache.org/repos/asf/tez/blob/d2d5f7eb/tez-ui/src/main/webapp/app/templates/vertex/index.hbs ---------------------------------------------------------------------- diff --git a/tez-ui/src/main/webapp/app/templates/vertex/index.hbs b/tez-ui/src/main/webapp/app/templates/vertex/index.hbs index 56f4eb9..437b381 100644 --- a/tez-ui/src/main/webapp/app/templates/vertex/index.hbs +++ b/tez-ui/src/main/webapp/app/templates/vertex/index.hbs @@ -74,6 +74,10 @@ + Progress + {{bs-progress-animated progressDecimal=progress}} + + {{t 'common.time.start'}} {{formatUnixTimestamp startTime}}