Return-Path: X-Original-To: apmail-falcon-commits-archive@minotaur.apache.org Delivered-To: apmail-falcon-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2370817B27 for ; Mon, 30 Mar 2015 10:07:32 +0000 (UTC) Received: (qmail 37749 invoked by uid 500); 30 Mar 2015 10:07:25 -0000 Delivered-To: apmail-falcon-commits-archive@falcon.apache.org Received: (qmail 37711 invoked by uid 500); 30 Mar 2015 10:07:25 -0000 Mailing-List: contact commits-help@falcon.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.apache.org Delivered-To: mailing list commits@falcon.apache.org Received: (qmail 37702 invoked by uid 99); 30 Mar 2015 10:07:25 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Mar 2015 10:07:25 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 97BE8E07D6; Mon, 30 Mar 2015 10:07:25 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: suhasv@apache.org To: commits@falcon.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: falcon git commit: FALCON-1119 Instance logs option is not returning the log location. Contributed by Suhas Vasu Date: Mon, 30 Mar 2015 10:07:25 +0000 (UTC) Repository: falcon Updated Branches: refs/heads/master c8e46d161 -> 6f78180ff FALCON-1119 Instance logs option is not returning the log location. Contributed by Suhas Vasu Project: http://git-wip-us.apache.org/repos/asf/falcon/repo Commit: http://git-wip-us.apache.org/repos/asf/falcon/commit/6f78180f Tree: http://git-wip-us.apache.org/repos/asf/falcon/tree/6f78180f Diff: http://git-wip-us.apache.org/repos/asf/falcon/diff/6f78180f Branch: refs/heads/master Commit: 6f78180ff2cac78d17a8415af5177252c8566a6e Parents: c8e46d1 Author: Suhas Vasu Authored: Mon Mar 30 15:37:01 2015 +0530 Committer: Suhas Vasu Committed: Mon Mar 30 15:37:01 2015 +0530 ---------------------------------------------------------------------- CHANGES.txt | 3 +++ .../org/apache/falcon/logging/JobLogMover.java | 2 ++ .../org/apache/falcon/logging/LogProvider.java | 24 +++++++++++++++----- 3 files changed, 23 insertions(+), 6 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/falcon/blob/6f78180f/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index db7daca..c136a86 100755 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -123,6 +123,9 @@ Trunk (Unreleased) (Suhas vasu) BUG FIXES + FALCON-1119 Instance logs option is not returning the log location + (Suhas Vasu) + FALCON-1100 UI : Failed to load data. 404 not found (Pallavi Rao via Suhas Vasu) http://git-wip-us.apache.org/repos/asf/falcon/blob/6f78180f/oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ---------------------------------------------------------------------- diff --git a/oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java b/oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java index 243487e..ba669c8 100644 --- a/oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java +++ b/oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java @@ -72,6 +72,8 @@ public class JobLogMover { return 0; } + //Assumption is - Each wf run will have a directory + //the corresponding job logs are stored within the respective dir Path path = new Path(context.getLogDir() + "/" + String.format("%03d", context.getWorkflowRunId())); FileSystem fs = HadoopClientFactory.get().createProxiedFileSystem(path.toUri()); http://git-wip-us.apache.org/repos/asf/falcon/blob/6f78180f/oozie/src/main/java/org/apache/falcon/logging/LogProvider.java ---------------------------------------------------------------------- diff --git a/oozie/src/main/java/org/apache/falcon/logging/LogProvider.java b/oozie/src/main/java/org/apache/falcon/logging/LogProvider.java index 2e5dffb..bac421f 100644 --- a/oozie/src/main/java/org/apache/falcon/logging/LogProvider.java +++ b/oozie/src/main/java/org/apache/falcon/logging/LogProvider.java @@ -81,11 +81,11 @@ public final class LogProvider { EntityUtil.getLogPath(cluster, entity) + "/job-" + EntityUtil.fromUTCtoURIDate(instance.instance) + "/*"); - FileStatus[] runs = fs.globStatus(jobPath); - if (runs.length > 0) { - // this is the latest run, dirs are sorted in increasing - // order of runs - return runs[runs.length - 1].getPath().getName(); + //Assumption here is - Each wf run will have a directory + //the corresponding job logs are stored within the respective dir + Path maxRunPath = findMaxRunIdPath(fs, jobPath); + if (maxRunPath != null) { + return maxRunPath.getName(); } else { LOG.warn("No run dirs are available in logs dir: {}", jobPath); return "-"; @@ -145,7 +145,7 @@ public final class LogProvider { } private String getActionStatus(String fileName) { - return fileName.contains("_SUCCEEDED.log") ? "SUCCEEDED" : "FAILED"; + return fileName.matches("(.*)SUCCEEDED(.*).log") ? "SUCCEEDED" : "FAILED"; } private String getDFSbrowserUrl(String hdfsPath, String logPath, @@ -159,4 +159,16 @@ public final class LogProvider { private String getFormatedRunId(String runId) { return String.format("%03d", Integer.parseInt(runId)); } + + private Path findMaxRunIdPath(FileSystem fs, Path jobLogPath) throws IOException{ + // In case of multiple runs, dirs are sorted in increasing + // order of runs. If runId is not specified, return the max runId (whose dir exists) + Path maxRunIdPath = null; + for (FileStatus fileStatus : fs.globStatus(jobLogPath)) { + if (fs.isDirectory(fileStatus.getPath())) { + maxRunIdPath = fileStatus.getPath(); + } + } + return maxRunIdPath; + } }