incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasad B <prasadb0...@gmail.com>
Subject Re: Reg tables to be used with HCatStorer
Date Thu, 29 Mar 2012 17:25:54 GMT
Thanks Alan. The problem persists and the error message I get is:

"ERROR 1115: Unsupported type: 50  in Pig's schema".

The test script is a simple 3-line,

A = load '/etc/passwd' using PigStorage(':');
B = foreach A generate $0 as id;
STORE B INTO 'default.unix_users' USING
org.apache.hcatalog.pig.HCatStorer();

The table unix_names had been created prior to the run as:

$ /usr/local/hcat/bin/hcat -e "CREATE TABLE unix_users (line STRING) STORED
AS RCFILE"


and the query on mysql CLI shows the correct metadata for the table
"unix_users".

(a few fields deleted for clarity)

mysql> select * from TBLS;

| TBL_ID | DB_ID | OWNER   | SD_ID |   TBL_NAME |             TBL_TYPE |
+--------+-------------+---------------+-----------+--------------------+-----------------------------+
|          1 |        1 |      prasad |         1 |     unix_users |
MANAGED_TABLE |

1 row in set (0.00 sec)

mysql> select * from COLUMNS;

| SD_ID | COMMENT | COLUMN_NAME | TYPE_NAME | INTEGER_IDX |
+-------+---------+-------------+-----------+-------------+
|     1 | NULL    | line        | string    |           0 |

1 row in set (0.00 sec)


Could you please give any pointers ? Also copied below is part of the log
that indicates the error in input / output file validation.

thank you,
Prasad.


Pig Stack Trace
---------------
ERROR 1115: Unsupported type: 50  in Pig's schema

org.apache.pig.impl.plan.VisitorException: ERROR 1115:
<file hcat.sh, line 3, column 0> Output Location Validation Failed for:
'default.unix_users More info to follow:
Unsupported type: 50  in Pig's schema
        at
org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:75)
        at
org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
        at
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
        at
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
        at
org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
        at
org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
        at
org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
        at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:298)
        at org.apache.pig.PigServer.compilePp(PigServer.java:1360)
        at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1297)
        at org.apache.pig.PigServer.execute(PigServer.java:1289)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:360)
        at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:561)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)




On Wed, Mar 28, 2012 at 5:30 PM, Alan Gates <gates@hortonworks.com> wrote:

> You should be able to store to an unpartitioned table by passing no
> arguments to HCatStorer (despite what the docs say).
>
> Alan.
>
> On Mar 28, 2012, at 5:26 PM, Prasad B wrote:
>
> > The documentation on HCatStorer says:
> >
> > "For the USING clause, you can have two string arguments:
> >
> > The first string argument represents key/value pairs for partition. This
> is a *mandatory* argument. In the above example, month, date and hour are
> columns on which table is partitioned. The values for partition keys should
> NOT be quoted, even if the partition key is defined to be of string type."
> >
> > Since the first argument is mandatory, it implies that tables into which
> data is stored using HCatStorer have to be created with at least one
> partition. For eg., as
> >
> > CREATE TABLE table_name (line STRING);
> >
> > ALTER TABLE table_name ADD PARTITION (col="xxx")
> >
> > Is my understanding correct ? My script tried to store to an
> unpartitioned table, by specifying ' ' for the first argument, and the
> HCatBaseStorer complains about the type.
> > Prasad.
> >
> >
>
>

Mime
View raw message