hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-4997) Update fair scheduler to use pluggable auth provider
Date Tue, 13 Dec 2016 22:58:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-4997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746522#comment-15746522
] 

Sergey Shelukhin edited comment on YARN-4997 at 12/13/16 10:58 PM:
-------------------------------------------------------------------

this break the following piece of code in Hive that seems to be determining the default queue
(I am not familiar with all the APIs called here), because the listener interface is made
invisible while the setReloadListener API that it is passed into is still public.
{noformat}
AllocationFileLoaderService allocsLoader = new AllocationFileLoaderService();
    allocsLoader.init(conf);
    allocsLoader.setReloadListener(new AllocationFileLoaderService.Listener() {
      @Override
      public void onReload(AllocationConfiguration allocs) {
        allocConf.set(allocs);
      }
    });
    try {
      allocsLoader.reloadAllocations();
    } catch (Exception ex) {
      throw new IOException("Failed to load queue allocations", ex);
    }
    if (allocConf.get() == null) {
      allocConf.set(new AllocationConfiguration(conf));
    }
    QueuePlacementPolicy queuePolicy = allocConf.get().getPlacementPolicy();
    if (queuePolicy != null) {
      requestedQueue = queuePolicy.assignAppToQueue(requestedQueue, userName);
{noformat}

Can you recommend a way to utilize reloadAllocations and get the queue (or placement policy,
or allocConf) without invoking this? Or some other workaround. 
Or otherwise can you please change Listener back to public?

Checking the code in 2.6, it seems like we can getConfig from the loader, but whatever reloadAllocations
might derive from the placement policy element in the xml file is not accessible.


was (Author: sershe):
this break the following piece of code in Hive that seems to be determining the default queue
(I am not familiar with all the APIs called here), because the listener interface is made
invisible while the setReloadListener API that it is passed into is still public.
{noformat}
AllocationFileLoaderService allocsLoader = new AllocationFileLoaderService();
    allocsLoader.init(conf);
    allocsLoader.setReloadListener(new AllocationFileLoaderService.Listener() {
      @Override
      public void onReload(AllocationConfiguration allocs) {
        allocConf.set(allocs);
      }
    });
    try {
      allocsLoader.reloadAllocations();
    } catch (Exception ex) {
      throw new IOException("Failed to load queue allocations", ex);
    }
    if (allocConf.get() == null) {
      allocConf.set(new AllocationConfiguration(conf));
    }
    QueuePlacementPolicy queuePolicy = allocConf.get().getPlacementPolicy();
    if (queuePolicy != null) {
      requestedQueue = queuePolicy.assignAppToQueue(requestedQueue, userName);
{noformat}

Can you recommend a way to utilize reloadAllocations and get the queue (or placement policy,
or allocConf) without invoking this? Or some other workaround. 
Or otherwise can you please change Listener back to public?

> Update fair scheduler to use pluggable auth provider
> ----------------------------------------------------
>
>                 Key: YARN-4997
>                 URL: https://issues.apache.org/jira/browse/YARN-4997
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.8.0
>            Reporter: Daniel Templeton
>            Assignee: Tao Jie
>             Fix For: 3.0.0-alpha2
>
>         Attachments: YARN-4997-001.patch, YARN-4997-002.patch, YARN-4997-003.patch, YARN-4997-004.patch,
YARN-4997-005.patch, YARN-4997-006.patch, YARN-4997-007.patch, YARN-4997-008.patch, YARN-4997-009.patch,
YARN-4997-010.patch, YARN-4997-011.patch
>
>
> Now that YARN-3100 has made the authorization pluggable, it should be supported by the
fair scheduler.  YARN-3100 only updated the capacity scheduler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message