eagle-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (EAGLE-946) Refactor MRRunningJobApp & HadoopQueueApp
Date Tue, 14 Mar 2017 02:53:41 GMT

    [ https://issues.apache.org/jira/browse/EAGLE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923462#comment-15923462
] 

ASF GitHub Bot commented on EAGLE-946:
--------------------------------------

Github user qingwen220 commented on a diff in the pull request:

    https://github.com/apache/eagle/pull/869#discussion_r105818056
  
    --- Diff: eagle-jpm/eagle-jpm-util/src/main/java/org/apache/eagle/jpm/util/resourcefetch/RMResourceFetcher.java
---
    @@ -112,40 +110,77 @@ private String getMRFinishedJobURL(String lastFinishedTime) {
                     + lastFinishedTime + "&" + Constants.ANONYMOUS_PARAMETER;
         }
     
    -    private List<AppInfo> doFetchRunningApplicationsList(String urlString, Constants.CompressionType
compressionType) throws Exception {
    -        List<AppInfo> result = new ArrayList<>(0);
    -        InputStream is = null;
    +    private String getAccepedAppURL() {
    +        String baseUrl = URLUtil.removeTrailingSlash(selector.getSelectedUrl());
    +        return String.format("%s/%s?state=ACCEPTED&%s", baseUrl, Constants.V2_APPS_URL,
Constants.ANONYMOUS_PARAMETER);
    +    }
    +
    +    private List<AppInfo> doFetchRunningApplicationsList(Constants.JobType jobType,
    +                                                         Constants.CompressionType compressionType,
    +                                                         Object... parameter) throws
Exception {
    +        Map<String, AppInfo> result = new HashMap();
    +        List<AppInfo> apps = new ArrayList<>();
             try {
    -            checkUrl();
    -            LOG.info("Going to call yarn api to fetch running application list: " + urlString);
    -            is = InputStreamUtils.getInputStream(urlString, null, compressionType);
    -            final AppsWrapper appWrapper = OBJ_MAPPER.readValue(is, AppsWrapper.class);
    -            if (appWrapper != null && appWrapper.getApps() != null &&
appWrapper.getApps().getApp() != null) {
    -                result = appWrapper.getApps().getApp();
    +            selector.checkUrl();
    +            String urlString = getRunningJobURL(jobType, null, null);
    +            if (parameter.length == 0) {
    +                return doFetchApplicationsList(urlString, compressionType);
                 }
    -            return result;
    -        } finally {
    -            if (is != null) {
    -                try {
    -                    is.close();
    -                } catch (Exception e) {
    -                    LOG.warn("{}", e);
    -                }
    +
    +            int requests = (int) parameter[0];
    +            if (requests <= 1) {
    +                return doFetchApplicationsList(urlString, compressionType);
                 }
    +
    +            long interval = DateTimeUtil.ONEHOUR * 6 / (requests - 1);
    +            long currentTime = System.currentTimeMillis();
    +            long earliestTime = currentTime - DateTimeUtil.ONEHOUR * 6;
    --- End diff --
    
    Yes. thanks for reminding me. 


> Refactor MRRunningJobApp & HadoopQueueApp 
> ------------------------------------------
>
>                 Key: EAGLE-946
>                 URL: https://issues.apache.org/jira/browse/EAGLE-946
>             Project: Eagle
>          Issue Type: Improvement
>    Affects Versions: v0.5.0
>            Reporter: Zhao, Qingwen
>            Assignee: Zhao, Qingwen
>
> Requirements 
> 1. REST apis to the remote cluster should be called only once.
> 2. For each request, the fetch running apps should be limits 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message