incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Crobak <joec...@gmail.com>
Subject Re: pig schema empty - how to debug?
Date Mon, 23 Jul 2012 13:57:41 GMT
I've tried a trunk build, because I thought this might be solved by
HCATALOG-423. It seems like it's not, although it's a related problem. Has
anyone had success with serde-generated columns via pig? Am I just hitting
some edge case?

Thanks,
Joe

On Sun, Jul 15, 2012 at 3:36 PM, Joe Crobak <joecrow@gmail.com> wrote:

> I have a custom InputFormat/SerDe for Hive, and I'd like to access data
> via Pig. I was able to get the classpaths setup correctly so that I can
> view the table via the hcat command line, as well as load the table from
> pig. Unfortunately, the schema for the relation I load is empty. If I dump
> the data, I get back empty tuples. And I can't project any columns, because
> HCat says the field doesn't exist in the schema.
>
> Has anyone seen this problem before? What's the best way to debug? I've
> tried tweaking numerous logging settings, but I can't get the hcat server
> or pig to output any debug level logging from hcatalog. Do I need to run a
> trunk build with HCATALOG-68, or is there another way? Investigating the MR
> job, my serde and input format are definitely being used -- I can see INFO
> level logging statements from them in the task attempt logs.
>
> Here's an example session:
>
> grunt> checkins = LOAD 'checkins' USING
> org.apache.hcatalog.pig.HCatLoader();
> 2012-07-15 19:19:49,739 [main] INFO  hive.metastore - Trying to connect to
> metastore with URI thrift://localhost:9999
> 2012-07-15 19:19:49,780 [main] INFO  hive.metastore - Connected to
> metastore.
> grunt> describe checkins;
> checkins: {}
> grunt> checkins = LOAD 'default.checkins' USING
> org.apache.hcatalog.pig.HCatLoader();
> grunt> describe checkins;
> checkins: {}
> grunt> x = limit checkins 5
> >> ;
> grunt> dump x;
>
> ..snip..
> ()
> ()
> ()
> ()
> ()
> grunt>
>

Mime
View raw message