Mailing-List: contact hive-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hive-dev@hadoop.apache.org
Message-ID: <1468281321.1236731210477.JavaMail.jira@brutus>
Date: Tue, 10 Mar 2009 17:26:50 -0700 (PDT)
From: "Ashish Thusoo (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Subject: [jira] Updated: (HIVE-233) Inserting into a table from a map/reduce
 transform results in no data
In-Reply-To: <1720140370.1231963139608.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HIVE-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashish Thusoo updated HIVE-233:
-------------------------------

             Priority: Blocker  (was: Major)
    Affects Version/s: 0.3.0

Marking this as a blocker for 0.3 as we need to understand this better and see if there is a workaround.

> Inserting into a table from a map/reduce transform results in no data
> ---------------------------------------------------------------------
>
>                 Key: HIVE-233
>                 URL: https://issues.apache.org/jira/browse/HIVE-233
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.3.0
>            Reporter: Josh Ferguson
>            Priority: Blocker
>
> When attempting to run a query of the form 
> INSERT OVERWRITE TABLE table_name PARTITION ( ... ) SELECT TRANSFORM ... 
> The table 'table_name' ends up empty even when the SELECT statement returns valid results that can be loaded by hand.
> *My target table*
> CREATE TABLE percentiles
> (actor_id STRING, percentile INT, count INT) 
> PARTITIONED BY (account STRING, application STRING, dataset STRING, hour INT, span INT) 
> CLUSTERED BY (actor_id) INTO 32 BUCKETS 
> ROW FORMAT DELIMITED 
> COLLECTION ITEMS TERMINATED BY '44'
> MAP KEYS TERMINATED BY '58'
> STORED AS TEXTFILE;
> *The attempted query*
> INSERT OVERWRITE TABLE percentiles PARTITION ( account='cUU5T7y6DmdzMJFcFt3JDe', application='test', dataset='test', hour=341976, span=168 ) SELECT TRANSFORM(actor_id) USING '/Users/Josh/cluster/bin/percentiles.rb' AS (actor_id, percentile, count) FROM ( SELECT actor_id FROM activities CLUSTER BY actor_id ) actors;
> *The result in hadoop (which is 0 bytes)*
> $ hadoop fs -ls /user/hive/warehouse/percentiles/*/*/*/*/*
> -rw-r--r--   1 Josh supergroup          0 2009-01-14 11:19 /user/hive/warehouse/percentiles/account=cUU5T7y6DmdzMJFcFt3JDe/application=test/dataset=test/hour=341976/span=168/attempt_200901131908_0009_r_000000_0
> *Inserting into a temporary directory first*
> hive> INSERT OVERWRITE DIRECTORY 'hdfs://localhost:9000/tmp/hdfs_out' SELECT TRANSFORM(actor_id) USING '/Users/Josh/cluster/bin/percentiles.rb' AS (actor_id, percentile, count) FROM ( SELECT actor_id FROM activities CLUSTER BY actor_id ) actors;
> *The results in hadoop (8600 bytes)*
> $ hadoop fs -ls /tmp/hdfs_out
> Found 1 items
> -rw-r--r--   1 Josh supergroup       8600 2009-01-14 11:27 /tmp/hdfs_out/attempt_200901131908_0010_r_000000_0
> *Loading from the temporary directory into percentiles*
> hive> LOAD DATA INPATH 'hdfs://localhost:9000/tmp/hdfs_out' INTO TABLE percentiles PARTITION ( account='cUU5T7y6DmdzMJFcFt3JDe', application='test', dataset='test', hour=341976, span=168 ); 
> Loading data to table percentiles partition {account=cUU5T7y6DmdzMJFcFt3JDe, application=test, dataset=test, hour=341976, span=168}
> OK
> *The results in hadoop (8600 bytes)*
> $ hadoop fs -ls /user/hive/warehouse/percentiles/*/*/*/*/*
> -rw-r--r--   1 Josh supergroup       8600 2009-01-14 11:27 /user/hive/warehouse/percentiles/account=cUU5T7y6DmdzMJFcFt3JDe/application=test/dataset=test/hour=341976/span=168/attempt_200901131908_0010_r_000000_0
> So it works fine when loading via a LOAD DATA statement from a temporary directory but not in a query with an INSERT OVERWRITE statement.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.