Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3F8731160D for ; Thu, 19 Jun 2014 05:21:26 +0000 (UTC) Received: (qmail 31980 invoked by uid 500); 19 Jun 2014 05:21:25 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 31915 invoked by uid 500); 19 Jun 2014 05:21:25 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 31902 invoked by uid 500); 19 Jun 2014 05:21:25 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 31898 invoked by uid 99); 19 Jun 2014 05:21:25 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jun 2014 05:21:25 +0000 Date: Thu, 19 Jun 2014 05:21:25 +0000 (UTC) From: "Teng Yutong (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-6584) Add HiveHBaseTableSnapshotInputFormat MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Yutong updated HIVE-6584: ------------------------------ Attachment: HIVE-6584.5.patch hi, this patch is my current workaround when dealing with HBase snapshot. but in order to make this patch work, still some changes is needed on the HBase side (change the visible descriptor of mapreduce.TableMapReduceUitls.convertStringToScan and mapreduce.TableSnapshotInputFormat.TableSnapshotRegionSplit into public). Since there is no issue related to this in HBase JIRA, so i haven't create a patch for these changes. > Add HiveHBaseTableSnapshotInputFormat > ------------------------------------- > > Key: HIVE-6584 > URL: https://issues.apache.org/jira/browse/HIVE-6584 > Project: Hive > Issue Type: Improvement > Components: HBase Handler > Reporter: Nick Dimiduk > Assignee: Nick Dimiduk > Fix For: 0.14.0 > > Attachments: HIVE-6584.0.patch, HIVE-6584.1.patch, HIVE-6584.2.patch, HIVE-6584.3.patch, HIVE-6584.4.patch, HIVE-6584.5.patch > > > HBASE-8369 provided mapreduce support for reading from HBase table snapsopts. This allows a MR job to consume a stable, read-only view of an HBase table directly off of HDFS. Bypassing the online region server API provides a nice performance boost for the full scan. HBASE-10642 is backporting that feature to 0.94/0.96 and also adding a {{mapred}} implementation. Once that's available, we should add an input format. A follow-on patch could work out how to integrate this functionality into the StorageHandler, similar to how HIVE-6473 integrates the HFileOutputFormat into existing table definitions. -- This message was sent by Atlassian JIRA (v6.2#6252)