hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/Tutorial" by StevenWong
Date Thu, 21 Apr 2011 20:31:19 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/Tutorial" page has been changed by StevenWong.
The comment on this change is: Remove unnecessary verbiage from an example dynamic-partition
insert query..
http://wiki.apache.org/hadoop/Hive/Tutorial?action=diff&rev1=34&rev2=35

--------------------------------------------------

      hive> set hive.exec.dynamic.partition.mode=nonstrict;
      hive> FROM page_view_stg pvs
            INSERT OVERWRITE TABLE page_view PARTITION(dt, country) 
-                  SELECT * FROM (
-                    SELECT pvs.viewTime, pvs.userid, pvs.page_url, pvs.referrer_url, null,
null, pvs.ip, 
+                  SELECT pvs.viewTime, pvs.userid, pvs.page_url, pvs.referrer_url, null,
null, pvs.ip, 
-                           from_unixtimestamp(pvs.viewTime, 'yyyy-MM-dd') ds, pvs.country

+                         from_unixtimestamp(pvs.viewTime, 'yyyy-MM-dd') ds, pvs.country 
-                    DISTRIBUTE BY ds, country
+                  DISTRIBUTE BY ds, country;
-                    ) T;
- }}} This query will generate a MapReduce job rather than Map-only job. The inner-most SELECT-clause
will be converted to a plan to the mappers and the output will be distributed to the reducers
based on the value of (ds, country) pairs. The INSERT-clause will be converted to the plan
in the reducer which writes to the dynamic partitions. 
+ }}} This query will generate a MapReduce job rather than Map-only job. The SELECT-clause
will be converted to a plan to the mappers and the output will be distributed to the reducers
based on the value of (ds, country) pairs. The INSERT-clause will be converted to the plan
in the reducer which writes to the dynamic partitions. 
  
  == Inserting into local files ==
  In certain situations you would want to write the output into a local file so that you could
load it into an excel spreadsheet. This can be accomplished with the following command:

Mime
View raw message