Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E5330200B91 for ; Thu, 25 Aug 2016 07:03:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E3C5E160AC1; Thu, 25 Aug 2016 05:03:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 7A823160AB1 for ; Thu, 25 Aug 2016 07:03:02 +0200 (CEST) Received: (qmail 48160 invoked by uid 500); 25 Aug 2016 05:03:01 -0000 Mailing-List: contact commits-help@eagle.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@eagle.incubator.apache.org Delivered-To: mailing list commits@eagle.incubator.apache.org Received: (qmail 48147 invoked by uid 99); 25 Aug 2016 05:03:01 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Aug 2016 05:03:01 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 1BAA5C0B4C for ; Thu, 25 Aug 2016 05:03:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id CjZj9qAHvZra for ; Thu, 25 Aug 2016 05:02:47 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id CAC625F642 for ; Thu, 25 Aug 2016 05:02:44 +0000 (UTC) Received: (qmail 47875 invoked by uid 99); 25 Aug 2016 05:02:44 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Aug 2016 05:02:43 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id B287CE1075; Thu, 25 Aug 2016 05:02:43 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: qingwzhao@apache.org To: commits@eagle.incubator.apache.org Date: Thu, 25 Aug 2016 05:02:46 -0000 Message-Id: In-Reply-To: <48801fac20264fdf856a857f728fa0d3@git.apache.org> References: <48801fac20264fdf856a857f728fa0d3@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [4/4] incubator-eagle git commit: [minor] Change resourceFetch to resourcefetch archived-at: Thu, 25 Aug 2016 05:03:05 -0000 [minor] Change resourceFetch to resourcefetch Author: Zhao, Qingwen Closes #386. Project: http://git-wip-us.apache.org/repos/asf/incubator-eagle/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-eagle/commit/b5fd782c Tree: http://git-wip-us.apache.org/repos/asf/incubator-eagle/tree/b5fd782c Diff: http://git-wip-us.apache.org/repos/asf/incubator-eagle/diff/b5fd782c Branch: refs/heads/develop Commit: b5fd782c0968ce8dd499de1899464c8edbf50dfd Parents: 6f5f972 Author: Qingwen Zhao Authored: Thu Aug 25 11:45:23 2016 +0800 Committer: Qingwen Zhao Committed: Thu Aug 25 13:00:44 2016 +0800 ---------------------------------------------------------------------- .../util/resourceFetch/RMResourceFetcher.java | 189 ------------ .../jpm/util/resourceFetch/ResourceFetcher.java | 27 -- .../SparkHistoryServerResourceFetcher.java | 88 ------ .../connection/InputStreamUtils.java | 68 ----- .../util/resourceFetch/connection/JobUtils.java | 43 --- .../connection/URLConnectionUtils.java | 101 ------- .../resourceFetch/ha/AbstractURLSelector.java | 110 ------- .../util/resourceFetch/ha/HAURLSelector.java | 28 -- .../resourceFetch/ha/HAURLSelectorImpl.java | 109 ------- .../jpm/util/resourceFetch/model/AppInfo.java | 210 -------------- .../util/resourceFetch/model/Applications.java | 38 --- .../util/resourceFetch/model/AppsWrapper.java | 37 --- .../util/resourceFetch/model/ClusterInfo.java | 119 -------- .../resourceFetch/model/ClusterInfoWrapper.java | 35 --- .../resourceFetch/model/JobCounterGroup.java | 46 --- .../resourceFetch/model/JobCounterItem.java | 62 ---- .../util/resourceFetch/model/JobCounters.java | 46 --- .../resourceFetch/model/JobCountersWrapper.java | 35 --- .../jpm/util/resourceFetch/model/MRJob.java | 289 ------------------- .../util/resourceFetch/model/MRJobsWrapper.java | 37 --- .../jpm/util/resourceFetch/model/MRTask.java | 109 ------- .../util/resourceFetch/model/MRTaskAttempt.java | 136 --------- .../model/MRTaskAttemptWrapper.java | 37 --- .../resourceFetch/model/MRTaskAttempts.java | 39 --- .../jpm/util/resourceFetch/model/MRTasks.java | 40 --- .../resourceFetch/model/MRTasksWrapper.java | 37 --- .../jpm/util/resourceFetch/model/MrJobs.java | 39 --- .../resourceFetch/model/SparkApplication.java | 57 ---- .../model/SparkApplicationAttempt.java | 73 ----- .../model/SparkApplicationWrapper.java | 38 --- .../util/resourceFetch/model/SparkExecutor.java | 155 ---------- .../jpm/util/resourceFetch/model/SparkJob.java | 165 ----------- .../util/resourceFetch/model/SparkStage.java | 211 -------------- .../jpm/util/resourceFetch/model/SparkTask.java | 111 ------- .../model/SparkTaskInputMetrics.java | 46 --- .../resourceFetch/model/SparkTaskMetrics.java | 118 -------- .../model/SparkTaskShuffleReadMetrics.java | 82 ------ .../model/SparkTaskShuffleWriteMetrics.java | 55 ---- .../resourceFetch/model/TaskCounterGroup.java | 46 --- .../resourceFetch/model/TaskCounterItem.java | 44 --- .../util/resourceFetch/model/TaskCounters.java | 46 --- .../model/TaskCountersWrapper.java | 35 --- .../url/JobListServiceURLBuilderImpl.java | 52 ---- .../resourceFetch/url/ServiceURLBuilder.java | 21 -- .../SparkCompleteJobServiceURLBuilderImpl.java | 33 --- .../url/SparkJobServiceURLBuilderImpl.java | 31 -- .../jpm/util/resourceFetch/url/URLUtil.java | 33 --- .../util/resourcefetch/RMResourceFetcher.java | 189 ++++++++++++ .../jpm/util/resourcefetch/ResourceFetcher.java | 27 ++ .../SparkHistoryServerResourceFetcher.java | 88 ++++++ .../connection/InputStreamUtils.java | 68 +++++ .../util/resourcefetch/connection/JobUtils.java | 43 +++ .../connection/URLConnectionUtils.java | 101 +++++++ .../resourcefetch/ha/AbstractURLSelector.java | 110 +++++++ .../util/resourcefetch/ha/HAURLSelector.java | 28 ++ .../resourcefetch/ha/HAURLSelectorImpl.java | 109 +++++++ .../jpm/util/resourcefetch/model/AppInfo.java | 210 ++++++++++++++ .../util/resourcefetch/model/Applications.java | 38 +++ .../util/resourcefetch/model/AppsWrapper.java | 37 +++ .../util/resourcefetch/model/ClusterInfo.java | 119 ++++++++ .../resourcefetch/model/ClusterInfoWrapper.java | 35 +++ .../resourcefetch/model/JobCounterGroup.java | 46 +++ .../resourcefetch/model/JobCounterItem.java | 62 ++++ .../util/resourcefetch/model/JobCounters.java | 46 +++ .../resourcefetch/model/JobCountersWrapper.java | 35 +++ .../jpm/util/resourcefetch/model/MRJob.java | 289 +++++++++++++++++++ .../util/resourcefetch/model/MRJobsWrapper.java | 37 +++ .../jpm/util/resourcefetch/model/MRTask.java | 109 +++++++ .../util/resourcefetch/model/MRTaskAttempt.java | 136 +++++++++ .../model/MRTaskAttemptWrapper.java | 37 +++ .../resourcefetch/model/MRTaskAttempts.java | 39 +++ .../jpm/util/resourcefetch/model/MRTasks.java | 40 +++ .../resourcefetch/model/MRTasksWrapper.java | 37 +++ .../jpm/util/resourcefetch/model/MrJobs.java | 39 +++ .../resourcefetch/model/SparkApplication.java | 57 ++++ .../model/SparkApplicationAttempt.java | 73 +++++ .../model/SparkApplicationWrapper.java | 38 +++ .../util/resourcefetch/model/SparkExecutor.java | 155 ++++++++++ .../jpm/util/resourcefetch/model/SparkJob.java | 165 +++++++++++ .../util/resourcefetch/model/SparkStage.java | 211 ++++++++++++++ .../jpm/util/resourcefetch/model/SparkTask.java | 111 +++++++ .../model/SparkTaskInputMetrics.java | 46 +++ .../resourcefetch/model/SparkTaskMetrics.java | 118 ++++++++ .../model/SparkTaskShuffleReadMetrics.java | 82 ++++++ .../model/SparkTaskShuffleWriteMetrics.java | 55 ++++ .../resourcefetch/model/TaskCounterGroup.java | 46 +++ .../resourcefetch/model/TaskCounterItem.java | 44 +++ .../util/resourcefetch/model/TaskCounters.java | 46 +++ .../model/TaskCountersWrapper.java | 35 +++ .../url/JobListServiceURLBuilderImpl.java | 52 ++++ .../resourcefetch/url/ServiceURLBuilder.java | 21 ++ .../SparkCompleteJobServiceURLBuilderImpl.java | 33 +++ .../url/SparkJobServiceURLBuilderImpl.java | 31 ++ .../jpm/util/resourcefetch/url/URLUtil.java | 33 +++ 94 files changed, 3606 insertions(+), 3606 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java deleted file mode 100644 index b1881ef..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/RMResourceFetcher.java +++ /dev/null @@ -1,189 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -/** - * - */ -package org.apache.eagle.jpm.util.resourcefetch; - -import org.apache.eagle.jpm.util.Constants; -import org.apache.eagle.jpm.util.resourcefetch.connection.InputStreamUtils; -import org.apache.eagle.jpm.util.resourcefetch.ha.HAURLSelector; -import org.apache.eagle.jpm.util.resourcefetch.ha.HAURLSelectorImpl; -import org.apache.eagle.jpm.util.resourcefetch.model.AppInfo; -import org.apache.eagle.jpm.util.resourcefetch.model.AppsWrapper; -import org.apache.eagle.jpm.util.resourcefetch.model.ClusterInfo; -import org.apache.eagle.jpm.util.resourcefetch.model.ClusterInfoWrapper; -import org.apache.eagle.jpm.util.resourcefetch.url.JobListServiceURLBuilderImpl; -import org.apache.eagle.jpm.util.resourcefetch.url.ServiceURLBuilder; -import org.apache.eagle.jpm.util.resourcefetch.url.SparkCompleteJobServiceURLBuilderImpl; -import org.apache.eagle.jpm.util.resourcefetch.url.URLUtil; -import org.codehaus.jackson.JsonParser; -import org.codehaus.jackson.map.ObjectMapper; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.io.InputStream; -import java.util.List; - -public class RMResourceFetcher implements ResourceFetcher { - - private static final Logger LOG = LoggerFactory.getLogger(RMResourceFetcher.class); - private final HAURLSelector selector; - private final ServiceURLBuilder jobListServiceURLBuilder; - private final ServiceURLBuilder sparkCompleteJobServiceURLBuilder; - private static final ObjectMapper OBJ_MAPPER = new ObjectMapper(); - - static { - OBJ_MAPPER.configure(JsonParser.Feature.ALLOW_NON_NUMERIC_NUMBERS, true); - } - - public RMResourceFetcher(String[] rmBasePaths) { - this.jobListServiceURLBuilder = new JobListServiceURLBuilderImpl(); - this.sparkCompleteJobServiceURLBuilder = new SparkCompleteJobServiceURLBuilderImpl(); - - this.selector = new HAURLSelectorImpl(rmBasePaths, jobListServiceURLBuilder, Constants.CompressionType.GZIP); - } - - private void checkUrl() throws IOException { - if (!selector.checkUrl(jobListServiceURLBuilder.build(selector.getSelectedUrl(), Constants.JobState.RUNNING.name()))) { - selector.reSelectUrl(); - } - } - - private List doFetchFinishApplicationsList(String urlString, Constants.CompressionType compressionType) throws Exception { - List result; - InputStream is = null; - try { - checkUrl(); - LOG.info("Going to call yarn api to fetch finished application list: " + urlString); - is = InputStreamUtils.getInputStream(urlString, null, compressionType); - final AppsWrapper appWrapper = OBJ_MAPPER.readValue(is, AppsWrapper.class); - if (appWrapper != null && appWrapper.getApps() != null - && appWrapper.getApps().getApp() != null) { - result = appWrapper.getApps().getApp(); - return result; - } - return null; - } finally { - if (is != null) { - try { - is.close(); - } catch (Exception e) { - LOG.warn("{}", e); - } - } - } - } - - private String getSparkRunningJobURL() { - return selector.getSelectedUrl() - + "/" - + Constants.V2_APPS_URL - + "?applicationTypes=SPARK&state=RUNNING&" - + Constants.ANONYMOUS_PARAMETER; - } - - private String getMRRunningJobURL() { - return String.format("%s/%s?applicationTypes=MAPREDUCE&state=RUNNING&%s", - selector.getSelectedUrl(), - Constants.V2_APPS_URL, - Constants.ANONYMOUS_PARAMETER); - } - - private String getMRFinishedJobURL(String lastFinishedTime) { - String url = URLUtil.removeTrailingSlash(selector.getSelectedUrl()); - return url + "/" + "Constants.V2_APPS_URL" - + "?applicationTypes=MAPREDUCE&state=FINISHED&finishedTimeBegin=" - + lastFinishedTime + "&" + Constants.ANONYMOUS_PARAMETER; - } - - private List doFetchRunningApplicationsList(String urlString, Constants.CompressionType compressionType) throws Exception { - List result; - InputStream is = null; - try { - checkUrl(); - LOG.info("Going to call yarn api to fetch running application list: " + urlString); - is = InputStreamUtils.getInputStream(urlString, null, compressionType); - final AppsWrapper appWrapper = OBJ_MAPPER.readValue(is, AppsWrapper.class); - if (appWrapper != null && appWrapper.getApps() != null && appWrapper.getApps().getApp() != null) { - result = appWrapper.getApps().getApp(); - return result; - } - return null; - } finally { - if (is != null) { - try { - is.close(); - } catch (Exception e) { - LOG.warn("{}", e); - } - } - } - } - - private List getResource(Constants.ResourceType resourceType, Constants.CompressionType compressionType, Object... parameter) throws Exception { - switch (resourceType) { - case COMPLETE_SPARK_JOB: - final String urlString = sparkCompleteJobServiceURLBuilder.build(selector.getSelectedUrl(), (String) parameter[0]); - return doFetchFinishApplicationsList(urlString, compressionType); - case RUNNING_SPARK_JOB: - return doFetchRunningApplicationsList(getSparkRunningJobURL(), compressionType); - case RUNNING_MR_JOB: - return doFetchRunningApplicationsList(getMRRunningJobURL(), compressionType); - case COMPLETE_MR_JOB: - return doFetchFinishApplicationsList(getMRFinishedJobURL((String) parameter[0]), compressionType); - default: - throw new Exception("Not support resourceType :" + resourceType); - } - } - - public List getResource(Constants.ResourceType resourceType, Object... parameter) throws Exception { - try { - return getResource(resourceType, Constants.CompressionType.GZIP, parameter); - } catch (java.util.zip.ZipException ex) { - return getResource(resourceType, Constants.CompressionType.NONE, parameter); - } - } - - private String getClusterInfoURL() { - return selector.getSelectedUrl() + "/" + Constants.YARN_API_CLUSTER_INFO + "?" + Constants.ANONYMOUS_PARAMETER; - } - - public ClusterInfo getClusterInfo() throws Exception { - InputStream is = null; - try { - checkUrl(); - final String urlString = getClusterInfoURL(); - LOG.info("Calling yarn api to fetch cluster info: " + urlString); - is = InputStreamUtils.getInputStream(urlString, null, Constants.CompressionType.GZIP); - final ClusterInfoWrapper clusterInfoWrapper = OBJ_MAPPER.readValue(is, ClusterInfoWrapper.class); - if (clusterInfoWrapper != null && clusterInfoWrapper.getClusterInfo() != null) { - return clusterInfoWrapper.getClusterInfo(); - } - return null; - } finally { - if (is != null) { - try { - is.close(); - } catch (Exception e) { - LOG.warn("{}", e); - } - } - } - } -} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ResourceFetcher.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ResourceFetcher.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ResourceFetcher.java deleted file mode 100644 index f920ddb..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ResourceFetcher.java +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch; - -import org.apache.eagle.jpm.util.Constants; - -import java.util.List; - -public interface ResourceFetcher { - //continue to refactor later - List getResource(Constants.ResourceType resoureType, Object... parameter) throws Exception; - -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/SparkHistoryServerResourceFetcher.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/SparkHistoryServerResourceFetcher.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/SparkHistoryServerResourceFetcher.java deleted file mode 100644 index ce2d9b8..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/SparkHistoryServerResourceFetcher.java +++ /dev/null @@ -1,88 +0,0 @@ -/* - * - * * Licensed to the Apache Software Foundation (ASF) under one or more - * * contributor license agreements. See the NOTICE file distributed with - * * this work for additional information regarding copyright ownership. - * * The ASF licenses this file to You under the Apache License, Version 2.0 - * * (the "License"); you may not use this file except in compliance with - * * the License. You may obtain a copy of the License at - * * - * * http://www.apache.org/licenses/LICENSE-2.0 - * * - * * Unless required by applicable law or agreed to in writing, software - * * distributed under the License is distributed on an "AS IS" BASIS, - * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * * See the License for the specific language governing permissions and - * * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch; - -import org.apache.eagle.jpm.util.Constants; -import org.apache.eagle.jpm.util.resourcefetch.connection.InputStreamUtils; -import org.apache.eagle.jpm.util.resourcefetch.model.SparkApplication; -import org.apache.eagle.jpm.util.resourcefetch.url.ServiceURLBuilder; -import org.apache.eagle.jpm.util.resourcefetch.url.SparkJobServiceURLBuilderImpl; -import org.apache.commons.codec.binary.Base64; -import org.codehaus.jackson.JsonParser; -import org.codehaus.jackson.map.ObjectMapper; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.FileNotFoundException; -import java.io.InputStream; -import java.util.Arrays; -import java.util.List; - -public class SparkHistoryServerResourceFetcher implements ResourceFetcher { - - private static final Logger LOG = LoggerFactory.getLogger(SparkHistoryServerResourceFetcher.class); - - private static final ObjectMapper OBJ_MAPPER = new ObjectMapper(); - - private String historyServerURL; - private final ServiceURLBuilder sparkDetailJobServiceURLBuilder; - private String auth; - - static { - OBJ_MAPPER.configure(JsonParser.Feature.ALLOW_NON_NUMERIC_NUMBERS, true); - } - - public SparkHistoryServerResourceFetcher(String historyServerURL, String userName, String pwd) { - this.historyServerURL = historyServerURL; - this.sparkDetailJobServiceURLBuilder = new SparkJobServiceURLBuilderImpl(); - this.auth = "Basic " + new String(new Base64().encode(String.format("%s:%s", userName, pwd).getBytes())); - ; - } - - private List doFetchSparkApplicationDetail(String appId) throws Exception { - InputStream is = null; - try { - final String urlString = sparkDetailJobServiceURLBuilder.build(this.historyServerURL, appId); - LOG.info("Going to call spark history server api to fetch spark job: " + urlString); - is = InputStreamUtils.getInputStream(urlString, auth, Constants.CompressionType.NONE); - SparkApplication app = OBJ_MAPPER.readValue(is, SparkApplication.class); - return Arrays.asList(app); - } catch (FileNotFoundException e) { - return null; - } finally { - if (is != null) { - try { - is.close(); - } catch (Exception e) { - LOG.warn("{}", e); - } - } - } - } - - public List getResource(Constants.ResourceType resoureType, Object... parameter) throws Exception { - switch (resoureType) { - case SPARK_JOB_DETAIL: - return doFetchSparkApplicationDetail((String) parameter[0]); - default: - throw new Exception("Not support resourceType :" + resoureType); - } - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/InputStreamUtils.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/InputStreamUtils.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/InputStreamUtils.java deleted file mode 100644 index e99e2aa..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/InputStreamUtils.java +++ /dev/null @@ -1,68 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.connection; - -import org.apache.eagle.jpm.util.Constants; - -import java.io.IOException; -import java.io.InputStream; -import java.net.URL; -import java.net.URLConnection; -import java.util.zip.GZIPInputStream; - -public class InputStreamUtils { - - private static final int CONNECTION_TIMEOUT = 10 * 1000; - private static final int READ_TIMEOUT = 5 * 60 * 1000; - private static final String GZIP_HTTP_HEADER = "Accept-Encoding"; - private static final String GZIP_COMPRESSION = "gzip"; - - private static InputStream openGZIPInputStream(URL url, String auth, int timeout) throws IOException { - final URLConnection connection = url.openConnection(); - connection.setConnectTimeout(CONNECTION_TIMEOUT); - connection.setReadTimeout(timeout); - connection.addRequestProperty(GZIP_HTTP_HEADER, GZIP_COMPRESSION); - if (null != auth) { - connection.setRequestProperty("Authorization", auth); - } - return new GZIPInputStream(connection.getInputStream()); - } - - private static InputStream openInputStream(URL url, String auth, int timeout) throws IOException { - URLConnection connection = url.openConnection(); - connection.setConnectTimeout(timeout); - if (null != auth) { - connection.setRequestProperty("Authorization", auth); - } - - return connection.getInputStream(); - } - - public static InputStream getInputStream(String urlString, String auth, Constants.CompressionType compressionType, int timeout) throws Exception { - final URL url = URLConnectionUtils.getUrl(urlString); - if (compressionType.equals(Constants.CompressionType.GZIP)) { - return openGZIPInputStream(url, auth, timeout); - } else { // CompressionType.NONE - return openInputStream(url, auth, timeout); - } - } - - public static InputStream getInputStream(String urlString, String auth, Constants.CompressionType compressionType) throws Exception { - return getInputStream(urlString, auth, compressionType, READ_TIMEOUT); - } - -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/JobUtils.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/JobUtils.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/JobUtils.java deleted file mode 100644 index 44d27e8..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/JobUtils.java +++ /dev/null @@ -1,43 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.connection; - -import org.apache.eagle.jpm.util.Constants; - -public class JobUtils { - - public static String checkAndAddLastSlash(String urlBase) { - if (!urlBase.endsWith("/")) { - return urlBase + "/"; - } - return urlBase; - } - - public static String getJobIDByAppID(String appID) { - if (appID.startsWith(Constants.APPLICATION_PREFIX)) { - return appID.replace(Constants.APPLICATION_PREFIX, Constants.JOB_PREFIX); - } - return null; - } - - public static String getAppIDByJobID(String jobID) { - if (jobID.startsWith(Constants.JOB_PREFIX)) { - return jobID.replace(Constants.JOB_PREFIX, Constants.APPLICATION_PREFIX); - } - return null; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/URLConnectionUtils.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/URLConnectionUtils.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/URLConnectionUtils.java deleted file mode 100644 index 2cd60ba..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/connection/URLConnectionUtils.java +++ /dev/null @@ -1,101 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.connection; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import java.io.IOException; -import java.net.MalformedURLException; -import java.net.URL; -import java.net.URLConnection; -import java.security.KeyManagementException; -import java.security.NoSuchAlgorithmException; -import java.security.cert.CertificateException; -import javax.net.ssl.*; - -public final class URLConnectionUtils { - //TODO: change some public method to private - private static final Logger LOG = LoggerFactory.getLogger(URLConnectionUtils.class); - - public static URLConnection getConnection(String url) throws Exception { - if (url.startsWith("https://")) { - return getHTTPSConnection(url); - } else if (url.startsWith("http://")) { - return getHTTPConnection(url); - } - throw new Exception("Invalid input argument url: " + url); - } - - public static URLConnection getHTTPConnection(String urlString) throws Exception { - final URL url = new URL(urlString); - return url.openConnection(); - } - - public static URL getUrl(String urlString) throws Exception { - if (urlString.toLowerCase().contains("https")) { - return getHTTPSUrl(urlString); - } else if (urlString.toLowerCase().contains("http")) { - return getURL(urlString); - } - throw new Exception("Invalid input argument url: " + urlString); - } - - public static URL getURL(String urlString) throws MalformedURLException { - return new URL(urlString); - } - - public static URL getHTTPSUrl(String urlString) throws MalformedURLException, NoSuchAlgorithmException, KeyManagementException { - // Create a trust manager that does not validate certificate chains - final TrustManager[] trustAllCerts = new TrustManager[] {new TrustAllX509TrustManager()}; - // Install the all-trusting trust manager - final SSLContext sc = SSLContext.getInstance("SSL"); - sc.init(null, trustAllCerts, new java.security.SecureRandom()); - HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory()); - // Create all-trusting host name verifier - final HostnameVerifier allHostsValid = new HostnameVerifier() { - public boolean verify(String hostname, SSLSession session) { - return true; - } - }; - HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid); - return new URL(urlString); - } - - public static URLConnection getHTTPSConnection(String urlString) throws IOException, KeyManagementException, NoSuchAlgorithmException { - final URL url = getHTTPSUrl(urlString); - return url.openConnection(); - } - - public static class TrustAllX509TrustManager implements X509TrustManager { - @Override - public void checkClientTrusted( - java.security.cert.X509Certificate[] chain, String authType) - throws CertificateException { - } - - @Override - public void checkServerTrusted( - java.security.cert.X509Certificate[] chain, String authType) - throws CertificateException { - } - - @Override - public java.security.cert.X509Certificate[] getAcceptedIssuers() { - return null; - } - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/AbstractURLSelector.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/AbstractURLSelector.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/AbstractURLSelector.java deleted file mode 100644 index d25d05b..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/AbstractURLSelector.java +++ /dev/null @@ -1,110 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch.ha; - -import org.apache.eagle.jpm.util.Constants; -import org.apache.eagle.jpm.util.resourcefetch.connection.InputStreamUtils; -import org.apache.hadoop.util.StringUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.io.InputStream; -import java.util.Arrays; - -public abstract class AbstractURLSelector implements HAURLSelector { - private final String[] urls; - private volatile String selectedUrl; - - private volatile boolean reselectInProgress; - private final Constants.CompressionType compressionType; - - private static final long MAX_RETRY_TIME = 3; - private static final Logger LOG = LoggerFactory.getLogger(HAURLSelectorImpl.class); - - public AbstractURLSelector(String[] urls, Constants.CompressionType compressionType) { - this.urls = urls; - this.compressionType = compressionType; - } - - public boolean checkUrl(String urlString) { - InputStream is = null; - try { - is = InputStreamUtils.getInputStream(urlString, null, compressionType); - } catch (Exception ex) { - LOG.info("get input stream from url: " + urlString + " failed. "); - return false; - } finally { - if (is != null) { - try { - is.close(); - } catch (IOException e) { - LOG.warn("{}", e); - } - } - } - return true; - } - - @Override - public String getSelectedUrl() { - if (selectedUrl == null) { - selectedUrl = urls[0]; - } - return selectedUrl; - } - - @Override - public void reSelectUrl() throws IOException { - if (reselectInProgress) { - return; - } - synchronized (this) { - if (reselectInProgress) { - return; - } - reselectInProgress = true; - try { - LOG.info("Going to reselect url"); - for (int i = 0; i < urls.length; i++) { - String urlToCheck = urls[i]; - LOG.info("Going to try url :" + urlToCheck); - for (int time = 0; time < MAX_RETRY_TIME; time++) { - if (checkUrl(buildTestURL(urlToCheck))) { - selectedUrl = urls[i]; - LOG.info("Successfully switch to new url : " + selectedUrl); - return; - } - LOG.info("try url " + urlToCheck + "fail for " + (time + 1) + " times, sleep 5 seconds before try again. "); - try { - Thread.sleep(5 * 1000); - } catch (InterruptedException ex) { - LOG.warn("{}", ex); - } - } - } - throw new IOException("No alive url found: " + StringUtils.join(";", Arrays.asList(this.urls))); - } finally { - reselectInProgress = false; - } - } - } - - protected abstract String buildTestURL(String urlToCheck); -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelector.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelector.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelector.java deleted file mode 100644 index fa9b52b..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelector.java +++ /dev/null @@ -1,28 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.ha; - -import java.io.IOException; - -public interface HAURLSelector { - - boolean checkUrl(String url); - - void reSelectUrl() throws IOException; - - String getSelectedUrl(); -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelectorImpl.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelectorImpl.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelectorImpl.java deleted file mode 100644 index a083ef2..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/ha/HAURLSelectorImpl.java +++ /dev/null @@ -1,109 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.ha; - -import org.apache.eagle.jpm.util.Constants; -import org.apache.eagle.jpm.util.resourcefetch.connection.InputStreamUtils; -import org.apache.eagle.jpm.util.resourcefetch.url.ServiceURLBuilder; -import org.apache.hadoop.util.StringUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.io.InputStream; -import java.util.Arrays; - -public class HAURLSelectorImpl implements HAURLSelector { - - private final String[] urls; - private volatile String selectedUrl; - private final ServiceURLBuilder builder; - - private volatile boolean reselectInProgress; - private final Constants.CompressionType compressionType; - private static final long MAX_RETRY_TIME = 2; - private static final Logger LOG = LoggerFactory.getLogger(HAURLSelectorImpl.class); - - public HAURLSelectorImpl(String[] urls, ServiceURLBuilder builder, Constants.CompressionType compressionType) { - this.urls = urls; - this.compressionType = compressionType; - this.builder = builder; - } - - public boolean checkUrl(String urlString) { - InputStream is = null; - try { - is = InputStreamUtils.getInputStream(urlString, null, compressionType); - } catch (Exception ex) { - LOG.info("get inputstream from url: " + urlString + " failed. "); - return false; - } finally { - if (is != null) { - try { - is.close(); - } catch (IOException e) { - LOG.warn("{}", e); - } - } - } - return true; - } - - @Override - public String getSelectedUrl() { - if (selectedUrl == null) { - selectedUrl = urls[0]; - } - return selectedUrl; - } - - @Override - public void reSelectUrl() throws IOException { - if (reselectInProgress) { - return; - } - synchronized (this) { - if (reselectInProgress) { - return; - } - reselectInProgress = true; - try { - LOG.info("Going to reselect url"); - for (int i = 0; i < urls.length; i++) { - String urlToCheck = urls[i]; - LOG.info("Going to try url :" + urlToCheck); - for (int time = 0; time < MAX_RETRY_TIME; time++) { - if (checkUrl(builder.build(urlToCheck, Constants.JobState.RUNNING.name()))) { - selectedUrl = urls[i]; - LOG.info("Successfully switch to new url : " + selectedUrl); - return; - } - LOG.info("try url " + urlToCheck + "fail for " + (time + 1) + " times, sleep 5 seconds before try again. "); - try { - Thread.sleep(1000); - } catch (InterruptedException ex) { - LOG.warn("{}", ex); - } - } - } - throw new IOException("No alive url found: " + StringUtils.join(";", Arrays.asList(this.urls))); - } finally { - reselectInProgress = false; - } - } - } -} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppInfo.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppInfo.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppInfo.java deleted file mode 100644 index f769217..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppInfo.java +++ /dev/null @@ -1,210 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -import java.io.Serializable; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class AppInfo implements Serializable { - private static final long serialVersionUID = 1L; - - private String id; - private String user; - private String name; - private String queue; - private String state; - private String finalStatus; - private double progress; - private String trackingUI; - private String trackingUrl; - private String diagnostics; - private String clusterId; - private String applicationType; - private long startedTime; - private long finishedTime; - private long elapsedTime; - private String amContainerLogs; - private String amHostHttpAddress; - private long allocatedMB; - private int allocatedVCores; - private int runningContainers; - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - public String getUser() { - return user; - } - - public void setUser(String user) { - this.user = user; - } - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public String getQueue() { - return queue; - } - - public void setQueue(String queue) { - this.queue = queue; - } - - public String getState() { - return state; - } - - public void setState(String state) { - this.state = state; - } - - public String getFinalStatus() { - return finalStatus; - } - - public void setFinalStatus(String finalStatus) { - this.finalStatus = finalStatus; - } - - public double getProgress() { - return progress; - } - - public void setProgress(double progress) { - this.progress = progress; - } - - public String getTrackingUI() { - return trackingUI; - } - - public void setTrackingUI(String trackingUI) { - this.trackingUI = trackingUI; - } - - public String getTrackingUrl() { - return trackingUrl; - } - - public void setTrackingUrl(String trackingUrl) { - this.trackingUrl = trackingUrl; - } - - public String getDiagnostics() { - return diagnostics; - } - - public void setDiagnostics(String diagnostics) { - this.diagnostics = diagnostics; - } - - public String getClusterId() { - return clusterId; - } - - public void setClusterId(String clusterId) { - this.clusterId = clusterId; - } - - public String getApplicationType() { - return applicationType; - } - - public void setApplicationType(String applicationType) { - this.applicationType = applicationType; - } - - public long getStartedTime() { - return startedTime; - } - - public void setStartedTime(long startedTime) { - this.startedTime = startedTime; - } - - public long getFinishedTime() { - return finishedTime; - } - - public void setFinishedTime(long finishedTime) { - this.finishedTime = finishedTime; - } - - public long getElapsedTime() { - return elapsedTime; - } - - public void setElapsedTime(long elapsedTime) { - this.elapsedTime = elapsedTime; - } - - public String getAmContainerLogs() { - return amContainerLogs; - } - - public void setAmContainerLogs(String amContainerLogs) { - this.amContainerLogs = amContainerLogs; - } - - public String getAmHostHttpAddress() { - return amHostHttpAddress; - } - - public void setAmHostHttpAddress(String amHostHttpAddress) { - this.amHostHttpAddress = amHostHttpAddress; - } - - public long getAllocatedMB() { - return allocatedMB; - } - - public void setAllocatedMB(long allocatedMB) { - this.allocatedMB = allocatedMB; - } - - public int getAllocatedVCores() { - return allocatedVCores; - } - - public void setAllocatedVCores(int allocatedVCores) { - this.allocatedVCores = allocatedVCores; - } - - public int getRunningContainers() { - return runningContainers; - } - - public void setRunningContainers(int runningContainers) { - this.runningContainers = runningContainers; - } - -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/Applications.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/Applications.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/Applications.java deleted file mode 100644 index 22ee2b9..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/Applications.java +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -import java.util.List; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class Applications { - - private List app; - - public List getApp() { - return app; - } - - public void setApp(List app) { - this.app = app; - } - -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppsWrapper.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppsWrapper.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppsWrapper.java deleted file mode 100644 index 63e37eb..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/AppsWrapper.java +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class AppsWrapper { - private static final long serialVersionUID = 1L; - - private Applications apps; - - public Applications getApps() { - return apps; - } - - public void setApps(Applications apps) { - this.apps = apps; - } - -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfo.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfo.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfo.java deleted file mode 100644 index 6e16b7f..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfo.java +++ /dev/null @@ -1,119 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -import java.io.Serializable; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class ClusterInfo implements Serializable { - private static final long serialVersionUID = 1L; - - private long id; - private long startedOn; - private String state; - private String haState; - private String resourceManagerVersion; - private String resourceManagerBuildVersion; - private String resourceManagerVersionBuiltOn; - private String hadoopVersion; - private String hadoopBuildVersion; - private String hadoopVersionBuiltOn; - - public long getId() { - return id; - } - - public void setId(long id) { - this.id = id; - } - - public long getStartedOn() { - return startedOn; - } - - public void setStartedOn(long startedOn) { - this.startedOn = startedOn; - } - - public String getState() { - return state; - } - - public void setState(String state) { - this.state = state; - } - - public String getHaState() { - return haState; - } - - public void setHaState(String haState) { - this.haState = haState; - } - - public String getResourceManagerVersion() { - return resourceManagerVersion; - } - - public void setResourceManagerVersion(String resourceManagerVersion) { - this.resourceManagerVersion = resourceManagerVersion; - } - - public String getResourceManagerBuildVersion() { - return resourceManagerBuildVersion; - } - - public void setResourceManagerBuildVersion(String resourceManagerBuildVersion) { - this.resourceManagerBuildVersion = resourceManagerBuildVersion; - } - - public String getResourceManagerVersionBuiltOn() { - return resourceManagerVersionBuiltOn; - } - - public void setResourceManagerVersionBuiltOn(String resourceManagerVersionBuiltOn) { - this.resourceManagerVersionBuiltOn = resourceManagerVersionBuiltOn; - } - - public String getHadoopVersion() { - return hadoopVersion; - } - - public void setHadoopVersion(String hadoopVersion) { - this.hadoopVersion = hadoopVersion; - } - - public String getHadoopBuildVersion() { - return hadoopBuildVersion; - } - - public void setHadoopBuildVersion(String hadoopBuildVersion) { - this.hadoopBuildVersion = hadoopBuildVersion; - } - - public String getHadoopVersionBuiltOn() { - return hadoopVersionBuiltOn; - } - - public void setHadoopVersionBuiltOn(String hadoopVersionBuiltOn) { - this.hadoopVersionBuiltOn = hadoopVersionBuiltOn; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfoWrapper.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfoWrapper.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfoWrapper.java deleted file mode 100644 index 4359e66..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/ClusterInfoWrapper.java +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class ClusterInfoWrapper { - private ClusterInfo clusterInfo; - - public ClusterInfo getClusterInfo() { - return clusterInfo; - } - - public void setClusterInfo(ClusterInfo clusterInfo) { - this.clusterInfo = clusterInfo; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterGroup.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterGroup.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterGroup.java deleted file mode 100644 index 189767b..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterGroup.java +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -import java.util.List; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class JobCounterGroup { - private String counterGroupName; - private List counter; - - public String getCounterGroupName() { - return counterGroupName; - } - - public void setCounterGroupName(String counterGroupName) { - this.counterGroupName = counterGroupName; - } - - public List getCounter() { - return counter; - } - - public void setCounter(List counter) { - this.counter = counter; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterItem.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterItem.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterItem.java deleted file mode 100644 index 001c5ff..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounterItem.java +++ /dev/null @@ -1,62 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class JobCounterItem { - private String name; - private long totalCounterValue; - private long mapCounterValue; - private long reduceCounterValue; - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public long getTotalCounterValue() { - return totalCounterValue; - } - - public void setTotalCounterValue(long totalCounterValue) { - this.totalCounterValue = totalCounterValue; - } - - public long getMapCounterValue() { - return mapCounterValue; - } - - public void setMapCounterValue(long mapCounterValue) { - this.mapCounterValue = mapCounterValue; - } - - public long getReduceCounterValue() { - return reduceCounterValue; - } - - public void setReduceCounterValue(long reduceCounterValue) { - this.reduceCounterValue = reduceCounterValue; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounters.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounters.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounters.java deleted file mode 100644 index cc0709f..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCounters.java +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -import java.util.List; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class JobCounters { - private String id; - private List counterGroup; - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - public List getCounterGroup() { - return counterGroup; - } - - public void setCounterGroup(List counterGroup) { - this.counterGroup = counterGroup; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCountersWrapper.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCountersWrapper.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCountersWrapper.java deleted file mode 100644 index f6bb29c..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/JobCountersWrapper.java +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class JobCountersWrapper { - public JobCounters getJobCounters() { - return jobCounters; - } - - public void setJobCounters(JobCounters jobCounters) { - this.jobCounters = jobCounters; - } - - private JobCounters jobCounters; -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJob.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJob.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJob.java deleted file mode 100644 index f12a5d7..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJob.java +++ /dev/null @@ -1,289 +0,0 @@ -/* - * - * * Licensed to the Apache Software Foundation (ASF) under one or more - * * contributor license agreements. See the NOTICE file distributed with - * * this work for additional information regarding copyright ownership. - * * The ASF licenses this file to You under the Apache License, Version 2.0 - * * (the "License"); you may not use this file except in compliance with - * * the License. You may obtain a copy of the License at - * * - * * http://www.apache.org/licenses/LICENSE-2.0 - * * - * * Unless required by applicable law or agreed to in writing, software - * * distributed under the License is distributed on an "AS IS" BASIS, - * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * * See the License for the specific language governing permissions and - * * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class MRJob { - private long startTime; - private long finishTime; - private long elapsedTime; - private String id; - private String name; - private String user; - private String state; - private int mapsTotal; - private int mapsCompleted; - private int reducesTotal; - private int reducesCompleted; - private double mapProgress; - private double reduceProgress; - private int mapsPending; - private int mapsRunning; - private int reducesPending; - private int reducesRunning; - private boolean uberized; - private String diagnostics; - private int newReduceAttempts; - private int runningReduceAttempts; - private int failedReduceAttempts; - private int killedReduceAttempts; - private int successfulReduceAttempts; - private int newMapAttempts; - private int runningMapAttempts; - private int failedMapAttempts; - private int killedMapAttempts; - private int successfulMapAttempts; - - public long getStartTime() { - return startTime; - } - - public void setStartTime(long startTime) { - this.startTime = startTime; - } - - public long getFinishTime() { - return finishTime; - } - - public void setFinishTime(long finishTime) { - this.finishTime = finishTime; - } - - public long getElapsedTime() { - return elapsedTime; - } - - public void setElapsedTime(long elapsedTime) { - this.elapsedTime = elapsedTime; - } - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public String getUser() { - return user; - } - - public void setUser(String user) { - this.user = user; - } - - public String getState() { - return state; - } - - public void setState(String state) { - this.state = state; - } - - public int getMapsTotal() { - return mapsTotal; - } - - public void setMapsTotal(int mapsTotal) { - this.mapsTotal = mapsTotal; - } - - public int getMapsCompleted() { - return mapsCompleted; - } - - public void setMapsCompleted(int mapsCompleted) { - this.mapsCompleted = mapsCompleted; - } - - public int getReducesTotal() { - return reducesTotal; - } - - public void setReducesTotal(int reducesTotal) { - this.reducesTotal = reducesTotal; - } - - public int getReducesCompleted() { - return reducesCompleted; - } - - public void setReducesCompleted(int reducesCompleted) { - this.reducesCompleted = reducesCompleted; - } - - public double getMapProgress() { - return mapProgress; - } - - public void setMapProgress(double mapProgress) { - this.mapProgress = mapProgress; - } - - public double getReduceProgress() { - return reduceProgress; - } - - public void setReduceProgress(double reduceProgress) { - this.reduceProgress = reduceProgress; - } - - public int getMapsPending() { - return mapsPending; - } - - public void setMapsPending(int mapsPending) { - this.mapsPending = mapsPending; - } - - public int getMapsRunning() { - return mapsRunning; - } - - public void setMapsRunning(int mapsRunning) { - this.mapsRunning = mapsRunning; - } - - public int getReducesPending() { - return reducesPending; - } - - public void setReducesPending(int reducesPending) { - this.reducesPending = reducesPending; - } - - public int getReducesRunning() { - return reducesRunning; - } - - public void setReducesRunning(int reducesRunning) { - this.reducesRunning = reducesRunning; - } - - public boolean isUberized() { - return uberized; - } - - public void setUberized(boolean uberized) { - this.uberized = uberized; - } - - public String getDiagnostics() { - return diagnostics; - } - - public void setDiagnostics(String diagnostics) { - this.diagnostics = diagnostics; - } - - public int getNewReduceAttempts() { - return newReduceAttempts; - } - - public void setNewReduceAttempts(int newReduceAttempts) { - this.newReduceAttempts = newReduceAttempts; - } - - public int getRunningReduceAttempts() { - return runningReduceAttempts; - } - - public void setRunningReduceAttempts(int runningReduceAttempts) { - this.runningReduceAttempts = runningReduceAttempts; - } - - public int getFailedReduceAttempts() { - return failedReduceAttempts; - } - - public void setFailedReduceAttempts(int failedReduceAttempts) { - this.failedReduceAttempts = failedReduceAttempts; - } - - public int getKilledReduceAttempts() { - return killedReduceAttempts; - } - - public void setKilledReduceAttempts(int killedReduceAttempts) { - this.killedReduceAttempts = killedReduceAttempts; - } - - public int getSuccessfulReduceAttempts() { - return successfulReduceAttempts; - } - - public void setSuccessfulReduceAttempts(int successfulReduceAttempts) { - this.successfulReduceAttempts = successfulReduceAttempts; - } - - public int getNewMapAttempts() { - return newMapAttempts; - } - - public void setNewMapAttempts(int newMapAttempts) { - this.newMapAttempts = newMapAttempts; - } - - public int getRunningMapAttempts() { - return runningMapAttempts; - } - - public void setRunningMapAttempts(int runningMapAttempts) { - this.runningMapAttempts = runningMapAttempts; - } - - public int getFailedMapAttempts() { - return failedMapAttempts; - } - - public void setFailedMapAttempts(int failedMapAttempts) { - this.failedMapAttempts = failedMapAttempts; - } - - public int getKilledMapAttempts() { - return killedMapAttempts; - } - - public void setKilledMapAttempts(int killedMapAttempts) { - this.killedMapAttempts = killedMapAttempts; - } - - public int getSuccessfulMapAttempts() { - return successfulMapAttempts; - } - - public void setSuccessfulMapAttempts(int successfulMapAttempts) { - this.successfulMapAttempts = successfulMapAttempts; - } -} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJobsWrapper.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJobsWrapper.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJobsWrapper.java deleted file mode 100644 index acd316f..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRJobsWrapper.java +++ /dev/null @@ -1,37 +0,0 @@ -/* - * - * * Licensed to the Apache Software Foundation (ASF) under one or more - * * contributor license agreements. See the NOTICE file distributed with - * * this work for additional information regarding copyright ownership. - * * The ASF licenses this file to You under the Apache License, Version 2.0 - * * (the "License"); you may not use this file except in compliance with - * * the License. You may obtain a copy of the License at - * * - * * http://www.apache.org/licenses/LICENSE-2.0 - * * - * * Unless required by applicable law or agreed to in writing, software - * * distributed under the License is distributed on an "AS IS" BASIS, - * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * * See the License for the specific language governing permissions and - * * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class MRJobsWrapper { - public MrJobs getJobs() { - return jobs; - } - - public void setJobs(MrJobs jobs) { - this.jobs = jobs; - } - - private MrJobs jobs; -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTask.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTask.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTask.java deleted file mode 100644 index 4a549de..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTask.java +++ /dev/null @@ -1,109 +0,0 @@ -/* - * - * * Licensed to the Apache Software Foundation (ASF) under one or more - * * contributor license agreements. See the NOTICE file distributed with - * * this work for additional information regarding copyright ownership. - * * The ASF licenses this file to You under the Apache License, Version 2.0 - * * (the "License"); you may not use this file except in compliance with - * * the License. You may obtain a copy of the License at - * * - * * http://www.apache.org/licenses/LICENSE-2.0 - * * - * * Unless required by applicable law or agreed to in writing, software - * * distributed under the License is distributed on an "AS IS" BASIS, - * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * * See the License for the specific language governing permissions and - * * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class MRTask { - private long startTime; - private long finishTime; - private long elapsedTime; - private double progress; - private String id; - private String state; - private String type; - private String successfulAttempt; - private String status; - - public long getStartTime() { - return startTime; - } - - public void setStartTime(long startTime) { - this.startTime = startTime; - } - - public long getFinishTime() { - return finishTime; - } - - public void setFinishTime(long finishTime) { - this.finishTime = finishTime; - } - - public long getElapsedTime() { - return elapsedTime; - } - - public void setElapsedTime(long elapsedTime) { - this.elapsedTime = elapsedTime; - } - - public double getProgress() { - return progress; - } - - public void setProgress(double progress) { - this.progress = progress; - } - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - public String getState() { - return state; - } - - public void setState(String state) { - this.state = state; - } - - public String getType() { - return type; - } - - public void setType(String type) { - this.type = type; - } - - public String getSuccessfulAttempt() { - return successfulAttempt; - } - - public void setSuccessfulAttempt(String successfulAttempt) { - this.successfulAttempt = successfulAttempt; - } - - public String getStatus() { - return status; - } - - public void setStatus(String status) { - this.status = status; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttempt.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttempt.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttempt.java deleted file mode 100644 index 218d021..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttempt.java +++ /dev/null @@ -1,136 +0,0 @@ -/* - * - * * Licensed to the Apache Software Foundation (ASF) under one or more - * * contributor license agreements. See the NOTICE file distributed with - * * this work for additional information regarding copyright ownership. - * * The ASF licenses this file to You under the Apache License, Version 2.0 - * * (the "License"); you may not use this file except in compliance with - * * the License. You may obtain a copy of the License at - * * - * * http://www.apache.org/licenses/LICENSE-2.0 - * * - * * Unless required by applicable law or agreed to in writing, software - * * distributed under the License is distributed on an "AS IS" BASIS, - * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * * See the License for the specific language governing permissions and - * * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class MRTaskAttempt { - private long startTime; - private long finishTime; - private long elapsedTime; - private double progress; - private String id; - private String rack; - private String state; - private String status; - private String nodeHttpAddress; - private String diagnostics; - private String type; - private String assignedContainerId; - - public long getStartTime() { - return startTime; - } - - public void setStartTime(long startTime) { - this.startTime = startTime; - } - - public long getFinishTime() { - return finishTime; - } - - public void setFinishTime(long finishTime) { - this.finishTime = finishTime; - } - - public long getElapsedTime() { - return elapsedTime; - } - - public void setElapsedTime(long elapsedTime) { - this.elapsedTime = elapsedTime; - } - - public double getProgress() { - return progress; - } - - public void setProgress(double progress) { - this.progress = progress; - } - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - public String getRack() { - return rack; - } - - public void setRack(String rack) { - this.rack = rack; - } - - public String getState() { - return state; - } - - public void setState(String state) { - this.state = state; - } - - public String getStatus() { - return status; - } - - public void setStatus(String status) { - this.status = status; - } - - public String getNodeHttpAddress() { - return nodeHttpAddress; - } - - public void setNodeHttpAddress(String nodeHttpAddress) { - this.nodeHttpAddress = nodeHttpAddress; - } - - public String getDiagnostics() { - return diagnostics; - } - - public void setDiagnostics(String diagnostics) { - this.diagnostics = diagnostics; - } - - public String getType() { - return type; - } - - public void setType(String type) { - this.type = type; - } - - public String getAssignedContainerId() { - return assignedContainerId; - } - - public void setAssignedContainerId(String assignedContainerId) { - this.assignedContainerId = assignedContainerId; - } -} http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/b5fd782c/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttemptWrapper.java ---------------------------------------------------------------------- diff --git a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttemptWrapper.java b/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttemptWrapper.java deleted file mode 100644 index 005b73a..0000000 --- a/eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourceFetch/model/MRTaskAttemptWrapper.java +++ /dev/null @@ -1,37 +0,0 @@ -/* - * - * * Licensed to the Apache Software Foundation (ASF) under one or more - * * contributor license agreements. See the NOTICE file distributed with - * * this work for additional information regarding copyright ownership. - * * The ASF licenses this file to You under the Apache License, Version 2.0 - * * (the "License"); you may not use this file except in compliance with - * * the License. You may obtain a copy of the License at - * * - * * http://www.apache.org/licenses/LICENSE-2.0 - * * - * * Unless required by applicable law or agreed to in writing, software - * * distributed under the License is distributed on an "AS IS" BASIS, - * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * * See the License for the specific language governing permissions and - * * limitations under the License. - * - */ - -package org.apache.eagle.jpm.util.resourcefetch.model; - -import org.codehaus.jackson.annotate.JsonIgnoreProperties; -import org.codehaus.jackson.map.annotate.JsonSerialize; - -@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL) -@JsonIgnoreProperties(ignoreUnknown = true) -public class MRTaskAttemptWrapper { - public MRTaskAttempts getTaskAttempts() { - return taskAttempts; - } - - public void setTaskAttempts(MRTaskAttempts taskAttempts) { - this.taskAttempts = taskAttempts; - } - - private MRTaskAttempts taskAttempts; -}