hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mich Talebzadeh" <m...@peridale.co.uk>
Subject Converting date format and excel money format in Hive table
Date Fri, 15 Jan 2016 10:15:24 GMT
Hi,

 

 

I am importing an excel sheet saved as csv file comma separated and compressed with bzip2
into Hive as external table with bzip2

 

The excel looks like this

 


Invoice Number

Payment date

Net

VAT

Total


360

10/02/2014

£10,000.00

£2000.00

£12,000.00

 

And the file (before bzip2) looks like this

 

Invoice Number,Payment date,Net,VAT,Total

360,10/02/2014,"▒12,000.00",▒2000.00,"▒12,000.00"

 

 

The external table is defined as

 

CREATE EXTERNAL TABLE stg_t2 (

INVOICENUMBER string

,PAYMENTDATE string

,NET string

,VAT string

,TOTAL string

)

COMMENT 'from csv file from excel sheet ‘

ROW FORMAT serde 'org.apache.hadoop.hive.serde2.OpenCSVSerde'

STORED AS TEXTFILE

LOCATION '/xyz/'

TBLPROPERTIES ("skip.header.line.count"="1")

;

 

 

And the table itself

 

 

CREATE TABLE t2 (

INVOICENUMBER          INT

,PAYMENTDATE            string

,NET                    string

,VAT                    string

,TOTAL                  string

)

COMMENT 'from csv file from excel sheet'

STORED AS ORC

TBLPROPERTIES ( "orc.compress"="ZLIB" )

;

INSERT INTO TABLE t2

SELECT

          INVOICENUMBER

        , PAYMENTDATE

        , NET

        , VAT

        , TOTAL

FROM

stg_t2;

 

 

Now the problem I have is that I do not seem to be able to convert PAYMENTDATE into timestamp
(from string) using CAST (PAYMENDATE AS TIMESTAMP) it RFETURNS NULL. Also I would like to
store currency properly  replacing “?”  with “£”?

 

+-------------------+-----------------+--------------+-------------+--------------+--+

| t2.invoicenumber  | t2.paymentdate  |    t2.net    |   t2.vat    |   t2.total   |

+-------------------+-----------------+--------------+-------------+--------------+--+

| 360               | 10/02/2014      | ?10,000.00   | ?2000.00    | ?12,000.00   |

|

 

Thanks

 

 

Dr Mich Talebzadeh

 

LinkedIn   <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

 

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy: Running the most Critical Financial Data on ASE 15

 <http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf

Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.


co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4

Publications due shortly:

Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly

 

 <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com

 

NOTE: The information in this email is proprietary and confidential. This message is for the
designated recipient only, if you are not the intended recipient, you should destroy it immediately.
Any information in this message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility
of the recipient to ensure that this email is virus free, therefore neither Peridale Technology
Ltd, its subsidiaries nor their employees accept any responsibility.

 


Mime
View raw message