Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D494710D08 for ; Tue, 17 Sep 2013 23:58:37 +0000 (UTC) Received: (qmail 45201 invoked by uid 500); 17 Sep 2013 23:58:37 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 45152 invoked by uid 500); 17 Sep 2013 23:58:37 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 45139 invoked by uid 99); 17 Sep 2013 23:58:37 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Sep 2013 23:58:37 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 2CA141D331F; Tue, 17 Sep 2013 23:58:34 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============4173624239389868242==" MIME-Version: 1.0 Subject: Re: Review Request 10857: Draft patch for review. Based on phase 1 mentioned in design doc. From: "Roshan Naik" To: "Roshan Naik" , "hive" , "Ashutosh Chauhan" Date: Tue, 17 Sep 2013 23:58:34 -0000 Message-ID: <20130917235834.31241.77997@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Roshan Naik" X-ReviewGroup: hive X-ReviewRequest-URL: https://reviews.apache.org/r/10857/ X-Sender: "Roshan Naik" References: <20130917235643.31242.67713@reviews.apache.org> In-Reply-To: <20130917235643.31242.67713@reviews.apache.org> Reply-To: "Roshan Naik" --===============4173624239389868242== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10857/ ----------------------------------------------------------- (Updated Sept. 17, 2013, 11:58 p.m.) Review request for hive. Changes ------- I have dropped the generated thrift files from this patch update for ease of review. Bugs: HIVE-5138 https://issues.apache.org/jira/browse/HIVE-5138 Repository: hive-git Description ------- Draft patch for review. based on phase 1 mentioned in design doc. Deviates slighlty from doc in the follow ways... 1) adds a couple of (temporary) rest calls to enable/disable streaming on a table. Later these will be replaced with support in DDL. 2) Also also HTTP methods are GET for easy testing with web browser 3) Authentication disabled on the new streaming HTTP methods Usage Examples on db named 'sdb' & table named 'log' : 1) Setup db & table with single partition column 'date': hcat -e "create database sdb; use sdb; create table log(msg string, region string) partitioned by (date string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' STORED AS TEXTFILE; " 2) To check streaming status: http://localhost:50111/templeton/v1/streaming/status?database=sdb&table=log 3) Enable Streaming: http://localhost:50111/templeton/v1/streaming/enable?database=sdb&table=log&col=date&value=1000 4) Get Chunk File to write to: http://localhost:50111/templeton/v1/streaming/chunkget?database=sdb&table=log&schema=blah&format=blah&record_separator=blah&field_separator=blah 5) Commit Chunk File: http://localhost:50111/templeton/v1/streaming/chunkcommit?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/2 6) Abort Chunk File: http://localhost:50111/templeton/v1/streaming/chunkabort?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/3 7) Roll Partition: http://localhost:50111/templeton/v1/streaming/partitionroll?database=sdb&table=log&partition_column=date&partition_value=3000 Diffs ----- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1af68a6 hcatalog/webhcat/svr/pom.xml 16a870e hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HcatStreamingDelegator.java PRE-CREATION hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Main.java 0f37278 hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java d8b6ce2 hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestStreaming.java PRE-CREATION hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/tool/HiveJdbcClient.java PRE-CREATION hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/tool/WebHCatClient.java PRE-CREATION metastore/if/hive_metastore.thrift 3fea057 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 39dda92 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java e165b2d metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java a2d6b1b metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a27243d metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java e410c3a metastore/src/model/org/apache/hadoop/hive/metastore/model/MTable.java 2a78ce9 metastore/src/model/package.jdo c42b5b0 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 8066784 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 0f9b16c metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 80fbbac Diff: https://reviews.apache.org/r/10857/diff/ Testing ------- Manual testing only Thanks, Roshan Naik --===============4173624239389868242==--