hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roshan Naik" <ros...@hortonworks.com>
Subject Review Request: Draft patch for review. Based on phase 1 mentioned in design doc.
Date Tue, 30 Apr 2013 02:41:53 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10857/
-----------------------------------------------------------

Review request for hive.


Description
-------

Draft patch for review. based on phase 1 mentioned in design doc. Deviates slighlty from doc
in the follow ways... 
1) adds a couple of (temporary) rest calls to enable/disable streaming on a table. Later these
will be replaced with support in DDL.
2) Also also HTTP methods are GET for easy testing with web browser
3) Authentication disabled on the new streaming HTTP methods


Usage Examples on db named 'sdb' & table named 'log' :

1) Setup db & table with single partition column 'date':
hcat -e "create database sdb; use sdb; create table log(msg string, region string) partitioned
by (date string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' STORED
AS TEXTFILE; "
2) To check streaming status:
http://localhost:50111/templeton/v1/streaming/status?database=sdb&table=log
3) Enable Streaming:
http://localhost:50111/templeton/v1/streaming/enable?database=sdb&table=log&col=date&value=1000
4) Get Chunk File to write to:
http://localhost:50111/templeton/v1/streaming/chunkget?database=sdb&table=log&schema=blah&format=blah&record_separator=blah&field_separator=blah
5) Commit Chunk File:
http://localhost:50111/templeton/v1/streaming/chunkcommit?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/2
6) Abort Chunk File:
http://localhost:50111/templeton/v1/streaming/chunkabort?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/3
7) Roll Partition:
http://localhost:50111/templeton/v1/streaming/partitionroll?database=sdb&table=log&partition_column=date&partition_value=3000


This addresses bug HIVE-4196.
    https://issues.apache.org/jira/browse/HIVE-4196


Diffs
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java c61d95b 
  hcatalog/webhcat/svr/src/main/java/org/apache/hcatalog/templeton/HcatStreamingDelegator.java
PRE-CREATION 
  hcatalog/webhcat/svr/src/main/java/org/apache/hcatalog/templeton/Server.java 29ac4b3 
  metastore/if/hive_metastore.thrift c2051f4 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 7b31d28 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 3d69472 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 3b90b44 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
d8d6e71 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java
030b54a 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
5929cda 
  metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php a69d214 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 6fd2cce 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 9b856e5 
  metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 25aa30c 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java dc14084 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java cef50f4 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java a2d6b1b 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 2079337 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 233fb46 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MTable.java 2a78ce9 
  metastore/src/model/package.jdo a84d2bf 
  metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 00eb0b4 

Diff: https://reviews.apache.org/r/10857/diff/


Testing
-------

Manual testing only


Thanks,

Roshan Naik


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message