hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8386) HCAT api call is case sensitive on fields in struct column
Date Mon, 13 Oct 2014 23:09:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170186#comment-14170186
] 

Sushanth Sowmyan commented on HIVE-8386:
----------------------------------------

Also, looking at JsonSerDe, that's not affected by HCatFieldSchema lowercasing, since it uses
that only for reading, and for writing, takes the types from the appropriate StructObjectInspector
passed in to the serialize method for it. If the AvroSerDe behaves similarly, HCatFieldSchema
doing the lowercasing should only affect reading, which is good. If not, we need to make that
more robust for definitions of case-sensitivity while working with hive.

> HCAT api call is case sensitive on fields in struct column
> ----------------------------------------------------------
>
>                 Key: HIVE-8386
>                 URL: https://issues.apache.org/jira/browse/HIVE-8386
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog, Types
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-8386.1.patch, HIVE-8386.2.patch, HIVE-8386.3.patch
>
>
> Falcon using hcat api to verify the target table schema and getting the error:
> {noformat}
> 2014-10-07 00:30:23,255 ERROR - [1972803970@qtp-1214921164-3:gfoetl:POST//entities/submitAndSchedule/feed
a0c221e3-efa8-4235-a403-b1047f23ec05] ~ Failure reason (FalconWebException:40)
> java.lang.RuntimeException: cannot find field givenName(lowercase form: givenname) in
[givenName, surname, middleName, gender, age, isGivenNameLowerCase, isGivenNameUpperCase,
isPrimary, isSurnameLowerCase, isSurnameUpperCase]
> 	at org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo.getStructFieldTypeInfo(StructTypeInfo.java:109)
> 	at org.apache.hive.hcatalog.data.schema.HCatSchemaUtils.constructHCatSchema(HCatSchemaUtils.java:154)
> 	at org.apache.hive.hcatalog.data.schema.HCatSchemaUtils.getHCatSchema(HCatSchemaUtils.java:165)
> 	at org.apache.hive.hcatalog.data.schema.HCatSchemaUtils.getHCatFieldSchema(HCatSchemaUtils.java:127)
> 	at org.apache.hive.hcatalog.data.schema.HCatSchemaUtils.getHCatFieldSchema(HCatSchemaUtils.java:115)
> 	at org.apache.hive.hcatalog.api.HCatTable.<init>(HCatTable.java:59)
> 	at org.apache.hive.hcatalog.api.HCatClientHMSImpl.getTable(HCatClientHMSImpl.java:157)
> 	at org.apache.falcon.catalog.HiveCatalogService.tableExists(HiveCatalogService.java:143)
> 	at org.apache.falcon.entity.parser.FeedEntityParser.validateStorageExists(FeedEntityParser.java:367)
> 	at org.apache.falcon.entity.parser.FeedEntityParser.validateFeedStorage(FeedEntityParser.java:309)
> 	at org.apache.falcon.entity.parser.FeedEntityParser.validate(FeedEntityParser.java:79)
> 	at org.apache.falcon.entity.parser.FeedEntityParser.validate(FeedEntityParser.java:54)
> 	at org.apache.falcon.resource.AbstractEntityManager.validate(AbstractEntityManager.java:364)
> 	at org.apache.falcon.resource.AbstractEntityManager.submitInternal(AbstractEntityManager.java:331)
> 	at org.apache.falcon.resource.AbstractEntityManager.submit(AbstractEntityManager.java:153)
> 	at org.apache.falcon.resource.ConfigSyncService.submit(ConfigSyncService.java:44)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:48)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$1.doExecute(SchedulableEntityManagerProxy.java:118)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:410)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.submit_aroundBody0(SchedulableEntityManagerProxy.java:120)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure1.run(SchedulableEntityManagerProxy.java:1)
> 	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> 	at org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.submit(SchedulableEntityManagerProxy.java:107)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.submit_aroundBody12(SchedulableEntityManagerProxy.java:341)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure13.run(SchedulableEntityManagerProxy.java:1)
> 	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> 	at org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.submitAndSchedule_aroundBody16(SchedulableEntityManagerProxy.java:341)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure17.run(SchedulableEntityManagerProxy.java:1)
> 	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> 	at org.apache.falcon.aspect.AbstractFalconAspect.logAround(AbstractFalconAspect.java:50)
> 	at org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.submitAndSchedule(SchedulableEntityManagerProxy.java:335)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> 	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> 	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> 	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> 	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> 	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> 	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> 	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> 	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> 	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> 	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> 	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> 	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> 	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> 	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> 	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> 	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> 	at org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:183)
> 	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:392)
> 	at org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:221)
> 	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> 	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> 	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> 	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> 	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> 	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> 	at org.mortbay.jetty.Server.handle(Server.java:326)
> 	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> 	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
> 	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
> 	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> 	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> 	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> 	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message