hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subramanyam Pattipaka (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-15807) MSCK operations hangs in HiveMetaStoreChecker.checkPartitionDirs
Date Fri, 03 Feb 2017 19:57:51 GMT

     [ https://issues.apache.org/jira/browse/HIVE-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Subramanyam Pattipaka updated HIVE-15807:
-----------------------------------------
    Attachment: msck-jstack.txt

> MSCK operations hangs in HiveMetaStoreChecker.checkPartitionDirs
> ----------------------------------------------------------------
>
>                 Key: HIVE-15807
>                 URL: https://issues.apache.org/jira/browse/HIVE-15807
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Subramanyam Pattipaka
>            Assignee: Pengcheng Xiong
>             Fix For: 2.2.0
>
>         Attachments: msck-jstack.txt
>
>
> The seems to be a regression from HIVE-14511. The operation was hung in checkPartitionDirs.
The data has 3 levels of partitions (month, date, id) which has total of 800 partitions.
> An example path would look like month=9/day=30/id=12
> The default value for hive config hive.mv.files.thread was set to 128. I have attached
the jstack of hive process used to run msck command
> checkPartitionDirs is implemented as recursive function which uses same pool to submit
worker threads. It seems thread pool ran out of thread to do the actual work and all threads
seems to be waiting and hung. Please take a look the stack and confirm if this is the case
here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message