hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manish Bhoge" <>
Subject Re: zip file or tar file cosumption
Date Wed, 26 Sep 2012 13:12:54 GMT
Hi Savant,

Got it. But I still need to understand that how to load zip? Can I directly use zip file in
external table. can u pls help to get the load statement. 
Sent from my BlackBerry, pls excuse typo

-----Original Message-----
From: "Savant, Keshav" <>
Date: Wed, 26 Sep 2012 12:25:38 
Subject: RE: zip file or tar file cosumption

Another solution would be

Using shell script do following

1.       unzip txt files,

2.       one by one merge those 50 (or N number of) text files into one text file,

3.       then the zip/tar that bigger text file,

4.       then that big zip/tar file can be uploaded into hive.

Keshav C Savant

From: Connell, Chuck []
Sent: Wednesday, September 26, 2012 4:04 PM
Subject: RE: zip file or tar file cosumption

This could be a problem. Hive uses newline as the record separator. A ZIP file will certainly
newline characters. So I doubt this is possible.

BUT, I would like to hear from anyone who has solved the "newline is always a record separator"
problem, because we ran into it for another type of compressed file.


From: Manish.Bhoge []
Sent: Wednesday, September 26, 2012 3:17 AM
Subject: zip file or tar file cosumption

I want to understand that would it be possible to utilize zip/tar files directly into Hive.
All the files has similar schema (structure).  Say 50 *.txt files are zipped into a single
zip file can we load data directly from this zip file OR should we need to unzip first?

Thanks & Regards
Manish Bhoge | Technical Architect  * Target DW/BI| * +919379850010 (M) Ext: 5691 VOIP: 22165
| * "Excellence is not a skill, It is an attitude." MySite<>

The information contained in this message is proprietary and/or confidential. If you are not
the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose,
distribute or use the message in any manner; and (iii) notify the sender immediately. In addition,
please be aware that any message addressed to our domain is subject to archiving and review
by persons other than the intended recipient. Thank you.

View raw message