atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <>
Subject [jira] [Commented] (ATLAS-464) HiveHook execution threads could crash due to a race condition
Date Tue, 02 Feb 2016 12:24:39 GMT


Hemanth Yamijala commented on ATLAS-464:

The reason this is happening is because, by default we launch with 5 executor threads that
are meant to asynchronously propagate Hive metadata to Atlas. Each thread on initialization
tries to check and register the Hive data model. Due to race condition, they could all detect
that the model is not registered and try to register it again. This causes the exception mentioned

> HiveHook execution threads could crash due to a race condition
> --------------------------------------------------------------
>                 Key: ATLAS-464
>                 URL:
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Hemanth Yamijala
>         Attachments: hive-hook-exception.txt
> Enable Atlas HiveHook in Hive with default config and an empty Atlas DB. Fire queries
fast to create tables - for e.g. by having table definitions in a file and executing with
hive -f <filename>. In the hive logs, we can see exceptions that hive types are being
redefined. The executor threads crash due to this exception. Sample of the exception log (will
attach the full log separately):
> {code}
> 2016-02-02 16:08:15,152 INFO  [Atlas Logger 4]: hook.HiveHook (
- Atlas hook failed
> org.apache.atlas.AtlasServiceException: Metadata service API CREATE_TYPE failed with
status 400(Bad Request) Response Body ({"error":"Redefinition of type hive_object_type not
supported","stackTrace":"org.apache.atlas.AtlasException: Redefinition of type hive_object_type
not supported\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.step1(\n\tat
org.apache.atlas.typesystem.types.TypeSystem.defineTypes(\n\tat org.apache.atlas.typesystem.types.TypeSystem.defineTypes(\n\tat\n\t
> {code}
> Thanks, [~ssainath] for finding this bug.

This message was sent by Atlassian JIRA

View raw message