hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao" <zsh...@gmail.com>
Subject Re: INSERT OVERWRITE not working with map/reduce transform
Date Tue, 13 Jan 2009 05:39:45 GMT
Should be tab separated.

if you run it without insert, is there any data on the screen?

Zheng

On Mon, Jan 12, 2009 at 9:32 PM, Josh Ferguson <josh@besquared.net> wrote:

> The only thing I could figure is that my output is incorrect.. Is the
> output from the transform script supposed to be tab separated or separated
> by the delimiters of the table you're trying to insert into? It doesn't seem
> to make any difference (my table is still empty no matter which one I try)
> but I'd better make sure just incase.
> Josh
>
> On Jan 12, 2009, at 9:11 PM, Zheng Shao wrote:
>
> Here are some examples:
>
> [zshao@xxx /hive.root] find ./ql/src/test/queries/clientpositive -name
> '*.q'  | xargs grep TRANSFORM
> ./ql/src/test/queries/clientpositive/input14_limit.q:  SELECT
> TRANSFORM(src.key, src.value)
> ./ql/src/test/queries/clientpositive/input14_limit.q:  SELECT
> TRANSFORM(src.key, src.value)
> ./ql/src/test/queries/clientpositive/input14.q:  SELECT TRANSFORM(src.key,
> src.value)
> ./ql/src/test/queries/clientpositive/input14.q:  SELECT TRANSFORM(src.key,
> src.value)
> ./ql/src/test/queries/clientpositive/input18.q:  SELECT TRANSFORM(src.key,
> src.value, 1+2, 3+4)
> ./ql/src/test/queries/clientpositive/input18.q:  SELECT TRANSFORM(src.key,
> src.value, 1+2, 3+4)
> ./ql/src/test/queries/clientpositive/scriptfile1.q:  SELECT
> TRANSFORM(src.key, src.value)
> ./ql/src/test/queries/clientpositive/input5.q:  SELECT
> TRANSFORM(src_thrift.lint, src_thrift.lintstring)
> ./ql/src/test/queries/clientpositive/input5.q:  SELECT
> TRANSFORM(src_thrift.lint, src_thrift.lintstring)
> ./ql/src/test/queries/clientpositive/input17.q:  SELECT
> TRANSFORM(src_thrift.aint + src_thrift.lint[0], src_thrift.lintstring[0])
> ./ql/src/test/queries/clientpositive/input17.q:  SELECT
> TRANSFORM(src_thrift.aint + src_thrift.lint[0], src_thrift.lintstring[0])
>
>
> On Mon, Jan 12, 2009 at 8:58 PM, Josh Ferguson <josh@besquared.net> wrote:
>
>> Anyone have any word on why this might not work? Can someone give me an
>> example of a query they use to INSERT OVERWRITE a table from a map and/or
>> reduce job that I could use as a reference?
>>
>> Josh F.
>>
>>
>> On Jan 11, 2009, at 9:48 PM, Josh Ferguson wrote:
>>
>>  I have a query that returns the proper results:
>>>
>>> SELECT TRANSFORM(actor_id) USING '/my/script.rb' AS (actor_id,
>>> percentile, count) FROM (SELECT actor_id FROM activities CLUSTER BY
>>> actor_id) actors;
>>>
>>> But when I do
>>>
>>> INSERT OVERWRITE TABLE percentiles
>>> SELECT TRANSFORM(actor_id) USING '/my/script.rb' AS (actor_id,
>>> percentile, count) FROM (SELECT actor_id FROM activities CLUSTER BY
>>> actor_id) actors;
>>>
>>> It says it loads data into the percentiles table but when I ask for data
>>> from that table I get:
>>>
>>> hive> SELECT actor_id, percentile, count FROM percentiles;
>>> FAILED: Error in semantic analysis:
>>> org.apache.hadoop.hive.ql.metadata.HiveException: Path
>>> /user/hive/warehouse/percentiles not a valid path
>>>
>>> $ hadoop fs -ls /user/hive/warehouse/percentiles/
>>> Found 1 items
>>> -rw-r--r--   1 Josh supergroup          0 2009-01-11 21:45
>>> /user/hive/warehouse/percentiles/attempt_200901112100_0010_r_000000_0
>>>
>>> It's nothing but an empty file.
>>>
>>> Am I doing something wrong?
>>>
>>> Josh Ferguson
>>>
>>
>>
>
>
> --
> Yours,
> Zheng
>
>
>


-- 
Yours,
Zheng

Mime
View raw message