atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharmadha Sainath (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-690) Read timed out exceptions when tables are imported into Atlas.
Date Wed, 04 May 2016 09:58:12 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270418#comment-15270418
] 

Sharmadha Sainath commented on ATLAS-690:
-----------------------------------------

With ATLAS-690-4.patch , test run with 10,000 tables on cluster, imported all tables without
any Read timed out exceptions. The import completed in 14 minutes 31 seconds and all tables
are imported into Atlas.  

> Read timed out exceptions when tables are imported into Atlas.
> --------------------------------------------------------------
>
>                 Key: ATLAS-690
>                 URL: https://issues.apache.org/jira/browse/ATLAS-690
>             Project: Atlas
>          Issue Type: Bug
>    Affects Versions: 0.7-incubating
>         Environment: Atlas with External Kafka/  HBase / Solr
> atlas.notification.hook.numthreads=5
> ATLAS_HOOK created with 5 partitions
>            Reporter: Sharmadha Sainath
>            Assignee: Hemanth Yamijala
>            Priority: Blocker
>             Fix For: 0.7-incubating
>
>         Attachments: ATLAS-690-3.patch, ATLAS-690-4.patch
>
>
> When 1000 tables are imported into Atlas using Hive hook,Read time out exceptions occur.
This happened with the latest Atlas build with commit id : 922a83c9a10e857d54855463225e9a5c375bc2b9.

>    • Hive ingestion was completed in 1 minute 50 secs. 
>    • Atlas ingestion took more than an hour .
> With Last 1000 tables run that was done in Atlas with commit id :
> b9575f29df3cc014f1b076abf52d88249bf4d0ef,
>  • Hive ingestion was completed in 3 minutes
>  • Atlas ingestion by 5 minutes.
> The Exception stack trace :
> Error handling message org.apache.atlas.notification.hook.HookNotification$EntityUpdateRequest@7474dd2d
(NotificationHookConsumer:224)
> com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read
timed out
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634)
> at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:911)
> at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:565)
> at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:935)
> at org.apache.atlas.AtlasClient.updateEntities(AtlasClient.java:530)
> at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:216)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:689)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1324)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message