atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suma Shivaprasad (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ATLAS-1051) Sqoop Hook does not package HDFS model jars which is required
Date Mon, 25 Jul 2016 18:48:20 GMT
Suma Shivaprasad created ATLAS-1051:
---------------------------------------

             Summary: Sqoop Hook does not package HDFS model jars which is required
                 Key: ATLAS-1051
                 URL: https://issues.apache.org/jira/browse/ATLAS-1051
             Project: Atlas
          Issue Type: Bug
            Reporter: Suma Shivaprasad
            Assignee: Suma Shivaprasad


Sqoop hook should package hdfs model jar and when sqoop commands to import are run, see this
error since hdfs model jar is missing resulting in errors while loading data.

{noformat}
2016-07-22 17:06:36,181 DEBUG [main]: ql.Driver (DriverContext.java:shutdown(132)) - Shutting
down query
LOAD DATA INPATH 'hdfs://localhost.localdomain:8020/user/hive/x_service' INTO TABLE `x_service`
2016-07-22 17:06:36,181 INFO  [main]: CliDriver (SessionState.java:printInfo(939)) - Time
taken: 4.791 seconds
2016-07-22 17:06:36,181 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(148)) -
<PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2016-07-22 17:06:36,181 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(176)) - </PERFLOG
method=releaseLocks start=1469232396181 end=1469232396181 duration=0 from=org.apache.hadoop.hive.ql.Driver>
2016-07-22 17:06:36,182 DEBUG [main]: session.SessionState (SessionState.java:close(1452))
- Removing resource dir /tmp/f60f6488-691f-440d-8fa0-01e26873f8a5_resources
2016-07-22 17:06:36,191 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115))
- ==> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path)
2016-07-22 17:06:36,192 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123))
- AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling pluginClassLoader.loadClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79))
- ==> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path)
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87))
- AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling pluginClassLoader.findClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97))
- AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling componentClassLoader.findClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133))
- AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling componentClassLoader.loadClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(142))
- <== AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): class org.apache.hadoop.fs.Path
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: hook.HiveHook (HiveHook.java:processHiveEntity(632))
- Registering DFS Path hdfs://localhost.localdomain:8020/user/hive/x_service
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115))
- ==> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes)
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123))
- AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes): calling pluginClassLoader.loadClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79))
- ==> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes)
2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87))
- AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes): calling pluginClassLoader.findClass()
2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97))
- AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes): calling componentClassLoader.findClass()
2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133))
- AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes): calling componentClassLoader.loadClass()
2016-07-22 17:06:36,197 ERROR [Atlas Logger 0]: hook.HiveHook (HiveHook.java:run(188)) - Atlas
hook failed due to error
java.lang.NoClassDefFoundError: org/apache/atlas/fs/model/FSDataTypes
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.fillHDFSDataSet(HiveMetaStoreBridge.java:556)
        at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:634)
        at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:599)
        at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:227)
        at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:82)
        at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:186)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.atlas.fs.model.FSDataTypes
        at java.lang.ClassLoader.findClass(ClassLoader.java:530)
        at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:333)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:137)
        ... 11 more
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message