atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-503) Not all Hive tables are not imported into Atlas when interrupted with search queries while importing.
Date Wed, 01 Jun 2016 13:01:04 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310253#comment-15310253
] 

Hemanth Yamijala commented on ATLAS-503:
----------------------------------------

Thanks for testing this, [~ssainath]

My hypothesis for the observations and subsequent recommendation for the db-cache-time is
as follows:

When we retry locks, we have started a transaction but have not yet closed. The worst case
wait time will be {{atlas.graph.storage.lock.retries x atlas.graph.storage.lock.wait-time}},
which for the said settings is 10 x 10000. The default value of {{atlas.graph.storage.cache.db-cache-time}}
is 10000 ms, which is too low, hence the observed error. Setting the value as said above seems
to be working for that reason.

I will do some testing from my end as well to confirm this behavior.

> Not all Hive tables are not imported into Atlas when interrupted with search queries
while importing.  
> -------------------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-503
>                 URL: https://issues.apache.org/jira/browse/ATLAS-503
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Sharmadha Sainath
>            Assignee: Hemanth Yamijala
>            Priority: Critical
>             Fix For: 0.7-incubating
>
>         Attachments: ATLAS-503.patch, hiv2atlaslogs.rtf
>
>
> On running a file containing 100 table creation commands using beeline -f , all hive
tables are created. But only 81 of them are imported into Atlas (HiveHook enabled) when queries
like "hive_table" is searched frequently while the import process for the table is going on.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message