hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8307) null character in columns.comments schema property breaks jobconf.xml
Date Mon, 05 Jan 2015 22:28:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265236#comment-14265236
] 

Ashutosh Chauhan commented on HIVE-8307:
----------------------------------------

I would argue that its a bug in {{MongoStorageHandler}} https://github.com/mongodb/mongo-hadoop/blob/master/hive/src/main/java/com/mongodb/hadoop/hive/MongoStorageHandler.java#L123

Methods {{configureInputJobProperties}} and {{configureOutputJobProperties}} are provided
in DefaultStorageHandler to copy few properties of interest from metadata (TableDesc) to runtime
(jobProperties) like uris etc. Blindly copying everything from that TableDesc::properties
may result in issues like these. 
Storage handlers are advised to only copy specific properties of interest, as metadata (TableDesc::properties)
map is not meant to be serializable.

> null character in columns.comments schema property breaks jobconf.xml
> ---------------------------------------------------------------------
>
>                 Key: HIVE-8307
>                 URL: https://issues.apache.org/jira/browse/HIVE-8307
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.13.0, 0.13.1
>            Reporter: Carl Laird
>
> It would appear that the fix for https://issues.apache.org/jira/browse/HIVE-6681 is causing
the null character to show up in job config xml files:
> I get the following when trying to insert into an elasticsearch backed table:
> [Fatal Error] :336:51: Character reference "&#
> 14/06/17 14:40:11 FATAL conf.Configuration: error parsing conf file: org.xml.sax.SAXParseException;
lineNumber: 336; columnNumber: 51; Character reference "&#
> Exception in thread "main" java.lang.RuntimeException: org.xml.sax.SAXParseException;
lineNumber: 336; columnNumber: 51; Character reference "&#
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1263)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1129)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1063)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:416)
> at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:604)
> at org.apache.hadoop.hive.conf.HiveConf.getBoolVar(HiveConf.java:1273)
> at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:667)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.xml.sax.SAXParseException; lineNumber: 336; columnNumber: 51; Character
reference "&#
> at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:251)
> at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:300)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1181)
> ... 11 more
> Execution failed with exit status: 1
> Line 336 of jobconf.xml:
> <property><name>columns.comments</name><value>&#0;&#0;&#0;&#0;&#0;&#0;&#0;&#0;&#0;&#0;&#0;&#0;</value></property>
> See https://groups.google.com/forum/#!msg/mongodb-user/lKbha0SzMP8/jvE8ZrJom4AJ for more
discussion.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message