hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject RegExSerde over JDBC... is it possible>
Date Mon, 30 Dec 2013 12:17:23 GMT
Hi folks..


Is there an JDBC or other API driven, programmatic way to launch a Hive Job
using SerDe's from the contrib library?

I've been attempting to run a query using Hive's JDBC driver which uses the
regex SerDe, however, it appears that hive doesn;t see this class.  As Im
launching the job from java via JDBC, im not sure where the class is not
found (i added it to HIVE_HOME/lib AND HADOOP_HOME/lib, so this is *really*
odd ).

2013-12-30 07:00:58.795 java[5829:1c03] Unable to load realm info from
SCDynamicStore
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is
not allowed in prolog.
Continuing ...
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/serde2/AbstractSerDe
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
    at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:410)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at com.sun.beans.finder.ClassFinder.findClass(ClassFinder.java:64)
    at com.sun.beans.finder.ClassFinder.findClass(ClassFinder.java:106)
    at com.sun.beans.finder.ClassFinder.resolveClass(ClassFinder.java:166)
    at
com.sun.beans.decoder.DocumentHandler.findClass(DocumentHandler.java:382)
    at
com.sun.beans.decoder.ClassElementHandler.getValue(ClassElementHandler.java:60)
    at
com.sun.beans.decoder.StringElementHandler.getValueObject(StringElementHandler.java:92)
    at
com.sun.beans.decoder.ElementHandler.endElement(ElementHandler.java:169)
    at
com.sun.beans.decoder.DocumentHandler.endElement(DocumentHandler.java:305)
    at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:606)
    at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1741)
    at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2898)
    at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:607)
    at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:488)
    at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:835)
    at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
    at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
    at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1210)
    at
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:568)
    at
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:302)
    at com.sun.beans.decoder.DocumentHandler.parse(DocumentHandler.java:356)
    at java.beans.XMLDecoder.parsingComplete(XMLDecoder.java:192)
    at java.beans.XMLDecoder.readObject(XMLDecoder.java:238)
    at
org.apache.hadoop.hive.ql.exec.Utilities.deserializeMapRedWork(Utilities.java:542)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:679)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hive.serde2.AbstractSerDe
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    ... 44 more


I've seen a JIRA alluding that custom serdes may *not* work in Hive JDBC...
Is that still the case https://issues.apache.org/jira/browse/HIVE-2685 ?

-- 
Jay Vyas
http://jayunit100.blogspot.com

Mime
View raw message