atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suma Shivaprasad (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (ATLAS-1050) Sqoop Hook does not package HDFS model jars which is required
Date Thu, 04 Aug 2016 23:28:20 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Suma Shivaprasad resolved ATLAS-1050.
-------------------------------------
    Resolution: Duplicate

> Sqoop Hook does not package HDFS model jars which is required
> -------------------------------------------------------------
>
>                 Key: ATLAS-1050
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1050
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>
> Sqoop hook should package hdfs model jar and when sqoop commands to import are run, see
this error since hdfs model jar is missing resulting in errors while loading data.
> {noformat}
> 2016-07-22 17:06:36,181 DEBUG [main]: ql.Driver (DriverContext.java:shutdown(132)) -
Shutting down query
> LOAD DATA INPATH 'hdfs://localhost.localdomain:8020/user/hive/x_service' INTO TABLE `x_service`
> 2016-07-22 17:06:36,181 INFO  [main]: CliDriver (SessionState.java:printInfo(939)) -
Time taken: 4.791 seconds
> 2016-07-22 17:06:36,181 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(148))
- <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
> 2016-07-22 17:06:36,181 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(176))
- </PERFLOG method=releaseLocks start=1469232396181 end=1469232396181 duration=0 from=org.apache.hadoop.hive.ql.Driver>
> 2016-07-22 17:06:36,182 DEBUG [main]: session.SessionState (SessionState.java:close(1452))
- Removing resource dir /tmp/f60f6488-691f-440d-8fa0-01e26873f8a5_resources
> 2016-07-22 17:06:36,191 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115))
- ==> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path)
> 2016-07-22 17:06:36,192 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123))
- AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling pluginClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79))
- ==> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path)
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87))
- AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling pluginClassLoader.findClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97))
- AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling componentClassLoader.findClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133))
- AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling componentClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(142))
- <== AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): class org.apache.hadoop.fs.Path
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: hook.HiveHook (HiveHook.java:processHiveEntity(632))
- Registering DFS Path hdfs://localhost.localdomain:8020/user/hive/x_service
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115))
- ==> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes)
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123))
- AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes): calling pluginClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79))
- ==> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes)
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87))
- AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes): calling pluginClassLoader.findClass()
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97))
- AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes): calling componentClassLoader.findClass()
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133))
- AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes): calling componentClassLoader.loadClass()
> 2016-07-22 17:06:36,197 ERROR [Atlas Logger 0]: hook.HiveHook (HiveHook.java:run(188))
- Atlas hook failed due to error
> java.lang.NoClassDefFoundError: org/apache/atlas/fs/model/FSDataTypes
>         at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.fillHDFSDataSet(HiveMetaStoreBridge.java:556)
>         at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:634)
>         at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:599)
>         at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:227)
>         at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:82)
>         at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.atlas.fs.model.FSDataTypes
>         at java.lang.ClassLoader.findClass(ClassLoader.java:530)
>         at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:333)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:137)
>         ... 11 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message