hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "YongHun Jeon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10721) The result does not show up after running hive query on Swift.
Date Thu, 19 Jun 2014 02:47:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

YongHun Jeon updated HADOOP-10721:
----------------------------------

    Priority: Major  (was: Critical)

> The result does not show up after running hive query on Swift.
> --------------------------------------------------------------
>
>                 Key: HADOOP-10721
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10721
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/swift
>            Reporter: YongHun Jeon
>
>  I configured Hadoop and Swift system as the site is mentioned : http://docs.openstack.org/developer/sahara/userdoc/hadoop-swift.html.
> So, I succeeded to access the Swift from Hadoop.
> I am running TPC-H performance test on Hadoop system integrated with Swift.
> I ran the below hive query.
> ---------------------------------------------------------------------------------------------
> DROP TABLE lineitem;
> DROP TABLE q1_pricing_summary_report;
> -- create tables and load data
> Create external table lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER
INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG
STRING, L_LINESTATUS STRING, L_SHIPDATE STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING,
L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED
BY '|' STORED AS TEXTFILE LOCATION 'swift://test.provider/tpch/lineitem';
> -- create the target table
> CREATE external TABLE q1_pricing_summary_report ( L_RETURNFLAG STRING, L_LINESTATUS STRING,
SUM_QTY DOUBLE, SUM_BASE_PRICE DOUBLE, SUM_DISC_PRICE DOUBLE, SUM_CHARGE DOUBLE, AVE_QTY DOUBLE,
AVE_PRICE DOUBLE, AVE_DISC DOUBLE, COUNT_ORDER INT) LOCATION 'swift://test.provider/user/result/q1_pricing_summary_report';
> set mapred.min.split.size=536870912;
> -- the query
> INSERT OVERWRITE TABLE q1_pricing_summary_report 
> SELECT 
>   L_RETURNFLAG, L_LINESTATUS, SUM(L_QUANTITY), SUM(L_EXTENDEDPRICE), SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)),
SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX)), AVG(L_QUANTITY), AVG(L_EXTENDEDPRICE), AVG(L_DISCOUNT),
COUNT(1) 
> FROM 
>   lineitem 
> WHERE 
>   L_SHIPDATE<='1998-09-02' 
> GROUP BY L_RETURNFLAG, L_LINESTATUS 
> ORDER BY L_RETURNFLAG, L_LINESTATUS;
> ---------------------------------------------------------------------------------------------
> You can get the files(such as lineitem) for the test through running dbgen which is in
this site : http://www.tpc.org/tpch/.
> I saw the some temporary files are generated and deleted. However, the result does not
show up after running hive query.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message