hawq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lav Jain (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HAWQ-1635) Cache Hadoop UGI for subsequent requests when PXF impersonation is on
Date Fri, 06 Jul 2018 19:39:00 GMT

     [ https://issues.apache.org/jira/browse/HAWQ-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lav Jain resolved HAWQ-1635.
----------------------------
       Resolution: Duplicate
    Fix Version/s: backlog

Didn't realize that there was already a Jira for this issue.

> Cache Hadoop UGI for subsequent requests when PXF impersonation is on	
> ----------------------------------------------------------------------
>
>                 Key: HAWQ-1635
>                 URL: https://issues.apache.org/jira/browse/HAWQ-1635
>             Project: Apache HAWQ
>          Issue Type: Improvement
>          Components: PXF
>            Reporter: Lav Jain
>            Assignee: Lav Jain
>            Priority: Major
>             Fix For: backlog
>
>
> To fix the memory leak issue as part of HAWQ-1621, we are closing the UGI reference at
the end of each PXF request. However, `FileSystem.closeAllForUGI` is an expensive operation
and is slowing down PXF almost by a factor of 4 (with impersonation on) compared with impersonation
turned off.
> The proposal is to cache the UGI object for each PXF query session. The getFragments()
call will create a new UGI and cache it using SegmentId:TransactionId. The bridge calls can
get this UGI from cache. The cache entry will be cleaned-up periodically based on some timeout
after inactivity.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message