Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
MIME-Version: 1.0
Date: Wed, 30 Dec 2015 18:43:41 +0800
Message-ID: 
 <CAHfzKEpwhnnRvp_iqqB_3n1QerLq_FYrj04B3Mq_6Lx4-DoKNw@mail.gmail.com>
Subject: how does Hive Partitioning works ?
From: Divya Gehlot <divya.htconex@gmail.com>
To: user@hive.apache.org
Content-Type: multipart/alternative; boundary=001a1135e154265b0b05281b36d9

--001a1135e154265b0b05281b36d9
Content-Type: text/plain; charset=UTF-8

Hi,
I am new bee to hive and trying to understand the hive partitioning .
My files are in CSV format
Steps which I followed
CREATE EXTERNAL TABLE IF NOT EXISTS loan_depo_part(COLUMN1 String ,COLUMN2
String ,COLUMN3 String ,
                                               COLUMN4 String,COLUMN5
String,COLUMN6 String,
                                               COLUMN7 Int ,COLUMN8 Int
,COLUMN9 String ,
                                               COLUMN10 String ,COLUMN11
String ,COLUMN12 String,
                                               COLUMN13 String ,COLUMN14
String ,
                                               COLUMN15 String ,COLUMN16
String ,
                                               COLUMN17 String ,COLUMN18
String ,
                                               COLUMN19 String ,COLUMN20
String ,
                                               COLUMN21 String ,COLUMN22
String )
COMMENT 'testing Partition'
PARTITIONED BY (Year String,Month String ,Day String)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
STORED AS TEXTFILE
TBLPROPERTIES ("skip.header.line.count"="1") ;


ALTER TABLE loan_depo_part ADD IF NOT EXISTS PARTITION(
Year=2015,Month=01,Day=01);

ALTER TABLE loan_depo_part PARTITION(Year=2015,Month=01,Day=01)
 SET LOCATION
'hdfs://namenode:8020/tmp/TestDivya/HiveInput/year=2015/month=01/day=01/';

Whereas my HDFS data location is
/TestDivya/HiveInput/year=2015/month=01/day=01/
I have few queries regarding the above partioning :
1. It creates the table when run the second step and gives the select
command it doesnt diplay any data
2. Do I need to create normal external table first and the partitioned one
next
 and then do the insert overwrite.

Basically I am not able to understand the partioning things mentioned above
I followed this link
<http://deanwampler.github.io/polyglotprogramming/papers/Hive-SQLforHadoop.pdf>

Would really appreciate the help/pointers.

Thanks,
Divya

--001a1135e154265b0b05281b36d9
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: base64

PGRpdiBkaXI9Imx0ciI+PGRpdj48ZGl2PjxkaXY+PGRpdj48ZGl2PjxkaXY+PGRpdj48ZGl2Pjxk
aXY+PGRpdj48ZGl2PkhpLDxicj48L2Rpdj5JIGFtIG5ldyBiZWUgdG8gaGl2ZSBhbmQgdHJ5aW5n
IHRvIHVuZGVyc3RhbmQgdGhlIGhpdmUgcGFydGl0aW9uaW5nIC48YnI+TXkgZmlsZXMgYXJlIGlu
IENTViBmb3JtYXQgPGJyPlN0ZXBzIHdoaWNoIEkgZm9sbG93ZWQgPGJyPkNSRUFURSBFWFRFUk5B
TCBUQUJMRSBJRiBOT1QgRVhJU1RTIGxvYW5fZGVwb19wYXJ0KENPTFVNTjEgU3RyaW5nICxDT0xV
TU4yIFN0cmluZyAsQ09MVU1OMyBTdHJpbmcgLDxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKg
wqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqAg
Q09MVU1ONCBTdHJpbmcsQ09MVU1ONSBTdHJpbmcsQ09MVU1ONiBTdHJpbmcsPGJyPsKgwqDCoCDC
oMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKg
wqDCoCDCoMKgwqAgwqDCoCBDT0xVTU43IEludCAsQ09MVU1OOCBJbnQgLENPTFVNTjkgU3RyaW5n
ICw8YnI+wqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKg
wqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgIENPTFVNTjEwIFN0cmluZyAsQ09MVU1OMTEg
U3RyaW5nICxDT0xVTU4xMiBTdHJpbmcsPGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDC
oMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoCBDT0xV
TU4xMyBTdHJpbmcgLENPTFVNTjE0IFN0cmluZyAsPGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKgIMKg
wqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDC
oCBDT0xVTU4xNSBTdHJpbmcgLENPTFVNTjE2IFN0cmluZyAsPGJyPsKgwqDCoCDCoMKgwqAgwqDC
oMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKg
wqAgwqDCoCBDT0xVTU4xNyBTdHJpbmcgLENPTFVNTjE4IFN0cmluZyAsPGJyPsKgwqDCoCDCoMKg
wqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDC
oCDCoMKgwqAgwqDCoCBDT0xVTU4xOSBTdHJpbmcgLENPTFVNTjIwIFN0cmluZyAsPGJyPsKgwqDC
oCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCDCoMKgwqAgwqDCoMKg
IMKgwqDCoCDCoMKgwqAgwqDCoCBDT0xVTU4yMSBTdHJpbmcgLENPTFVNTjIyIFN0cmluZyApPGJy
PkNPTU1FTlQgJiMzOTt0ZXN0aW5nIFBhcnRpdGlvbiYjMzk7PGJyPlBBUlRJVElPTkVEIEJZIChZ
ZWFyIFN0cmluZyxNb250aCBTdHJpbmcgLERheSBTdHJpbmcpPGJyPlJPVyBGT1JNQVQgREVMSU1J
VEVEIEZJRUxEUyBURVJNSU5BVEVEIEJZICYjMzk7LCYjMzk7PGJyPlNUT1JFRCBBUyBURVhURklM
RTxicj5UQkxQUk9QRVJUSUVTICgmcXVvdDtza2lwLmhlYWRlci5saW5lLmNvdW50JnF1b3Q7PSZx
dW90OzEmcXVvdDspIDs8YnI+PGJyPjxicj48YnI+QUxURVIgVEFCTEUgbG9hbl9kZXBvX3BhcnQg
QUREIElGIE5PVCBFWElTVFMgUEFSVElUSU9OKCBZZWFyPTIwMTUsTW9udGg9MDEsRGF5PTAxKTs8
YnI+PGJyPkFMVEVSIFRBQkxFIGxvYW5fZGVwb19wYXJ0IFBBUlRJVElPTihZZWFyPTIwMTUsTW9u
dGg9MDEsRGF5PTAxKTxicj7CoFNFVCBMT0NBVElPTiA8YnI+JiMzOTtoZGZzOi8vbmFtZW5vZGU6
ODAyMC90bXAvVGVzdERpdnlhL0hpdmVJbnB1dC95ZWFyPTIwMTUvbW9udGg9MDEvZGF5PTAxLyYj
Mzk7Ozxicj48L2Rpdj48YnI+PC9kaXY+PGRpdj5XaGVyZWFzIG15IEhERlMgZGF0YSBsb2NhdGlv
biBpcyAvVGVzdERpdnlhL0hpdmVJbnB1dC95ZWFyPTIwMTUvbW9udGg9MDEvZGF5PTAxLzwvZGl2
PkkgaGF2ZSBmZXcgcXVlcmllcyByZWdhcmRpbmcgdGhlIGFib3ZlIHBhcnRpb25pbmcgOjxicj48
L2Rpdj4xLiBJdCBjcmVhdGVzIHRoZSB0YWJsZSB3aGVuIHJ1biB0aGUgc2Vjb25kIHN0ZXAgYW5k
IGdpdmVzIHRoZSBzZWxlY3QgY29tbWFuZCBpdCBkb2VzbnQgZGlwbGF5IGFueSBkYXRhIDxicj48
L2Rpdj4yLiBEbyBJIG5lZWQgdG8gY3JlYXRlIG5vcm1hbCBleHRlcm5hbCB0YWJsZSBmaXJzdCBh
bmQgdGhlIHBhcnRpdGlvbmVkIG9uZSBuZXh0IDxicj48L2Rpdj7CoGFuZCB0aGVuIGRvIHRoZSBp
bnNlcnQgb3ZlcndyaXRlLjxicj48YnI+PC9kaXY+QmFzaWNhbGx5IEkgYW0gbm90IGFibGUgdG8g
dW5kZXJzdGFuZCB0aGUgcGFydGlvbmluZyB0aGluZ3MgbWVudGlvbmVkIGFib3ZlIDxicj48L2Rp
dj5JIGZvbGxvd2VkIHRoaXMgPGEgaHJlZj0iaHR0cDovL2RlYW53YW1wbGVyLmdpdGh1Yi5pby9w
b2x5Z2xvdHByb2dyYW1taW5nL3BhcGVycy9IaXZlLVNRTGZvckhhZG9vcC5wZGYiPmxpbmsgPC9h
Pjxicj48YnI+PC9kaXY+V291bGQgcmVhbGx5IGFwcHJlY2lhdGUgdGhlIGhlbHAvcG9pbnRlcnMu
PGJyPjxicj48L2Rpdj5UaGFua3MsPGJyPjwvZGl2PkRpdnlhPGJyPjxkaXY+PGRpdj48ZGl2Pjxk
aXY+PGRpdj48ZGl2PjxkaXY+PGRpdj48ZGl2PiA8ZGl2PjxkaXY+PGRpdj48ZGl2Pjxicj48L2Rp
dj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48
L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj4NCg==
--001a1135e154265b0b05281b36d9--