hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <>
Subject [jira] [Updated] (HIVE-12043) UGI instances being used in IO elevator threads are incorrect
Date Tue, 06 Oct 2015 21:09:27 GMT


Siddharth Seth updated HIVE-12043:
    Attachment: HIVE-12043.1.txt

Patch stores the UGI at object creation time, and then uses it during execution in a thread.

After this patch, the behaviour is to use a separate UGI instance for each task - with the
tokens associated with the task. (That would have been the previous behaviour as well - except
for the linking of the first UGI to a thread).

We can look at using a single UGI for all IO-elevator work as an alternate approach, with
SQL standard auth instead of storage auth.

[~sershe], [~gopalv] - please review.

> UGI instances being used in IO elevator threads are incorrect
> -------------------------------------------------------------
>                 Key: HIVE-12043
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>             Fix For: llap
>         Attachments: HIVE-12043.1.txt
> ... which leads to FileSystem closed exceptions.
> I'm not sure yet if this is a result of the threadpool being used, and UGI not working
well with threadpools, or something else.
> The UGI instance which was setup - at what looks to be thread creation time - ends up
being used for several different reads, ignoring the actual UGI passed in. At some point this
changes to a new incorrect UGI.
> A simple fix is to propagate the correct UGI all the way to the reader, and that fixes
the FileSystem Closed exception. Figuring out the precise reason would be good though.
> Related to HIVE-9898.

This message was sent by Atlassian JIRA

View raw message