hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roshan Naik" <ros...@hortonworks.com>
Subject Re: Review Request 10857: Draft patch for review. Based on phase 1 mentioned in design doc.
Date Tue, 17 Sep 2013 23:56:43 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10857/
-----------------------------------------------------------

(Updated Sept. 17, 2013, 11:56 p.m.)


Review request for hive.


Changes
-------

Updating patch. Addresses all comments captured so far form Brock, Allan and Eugene.
Also added unit tests.


Bugs: HIVE-4196
    https://issues.apache.org/jira/browse/HIVE-4196


Repository: hive-git


Description
-------

Draft patch for review. based on phase 1 mentioned in design doc. Deviates slighlty from doc
in the follow ways... 
1) adds a couple of (temporary) rest calls to enable/disable streaming on a table. Later these
will be replaced with support in DDL.
2) Also also HTTP methods are GET for easy testing with web browser
3) Authentication disabled on the new streaming HTTP methods


Usage Examples on db named 'sdb' & table named 'log' :

1) Setup db & table with single partition column 'date':
hcat -e "create database sdb; use sdb; create table log(msg string, region string) partitioned
by (date string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' STORED
AS TEXTFILE; "
2) To check streaming status:
http://localhost:50111/templeton/v1/streaming/status?database=sdb&table=log
3) Enable Streaming:
http://localhost:50111/templeton/v1/streaming/enable?database=sdb&table=log&col=date&value=1000
4) Get Chunk File to write to:
http://localhost:50111/templeton/v1/streaming/chunkget?database=sdb&table=log&schema=blah&format=blah&record_separator=blah&field_separator=blah
5) Commit Chunk File:
http://localhost:50111/templeton/v1/streaming/chunkcommit?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/2
6) Abort Chunk File:
http://localhost:50111/templeton/v1/streaming/chunkabort?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/3
7) Roll Partition:
http://localhost:50111/templeton/v1/streaming/partitionroll?database=sdb&table=log&partition_column=date&partition_value=3000


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1af68a6 
  hcatalog/webhcat/svr/pom.xml 16a870e 
  hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HcatStreamingDelegator.java
PRE-CREATION 
  hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Main.java 0f37278

  hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java d8b6ce2

  hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestStreaming.java
PRE-CREATION 
  hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/tool/HiveJdbcClient.java
PRE-CREATION 
  hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/tool/WebHCatClient.java
PRE-CREATION 
  metastore/if/hive_metastore.thrift 3fea057 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 39dda92 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java e165b2d 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java a2d6b1b 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a27243d 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java e410c3a 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MTable.java 2a78ce9 
  metastore/src/model/package.jdo c42b5b0 
  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 8066784

  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 0f9b16c

  metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 80fbbac 

Diff: https://reviews.apache.org/r/10857/diff/


Testing
-------

Manual testing only


Thanks,

Roshan Naik


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message