Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D3CC9F392 for ; Wed, 21 Aug 2013 15:42:20 +0000 (UTC) Received: (qmail 64215 invoked by uid 500); 21 Aug 2013 15:42:18 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 64089 invoked by uid 500); 21 Aug 2013 15:42:14 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 64076 invoked by uid 99); 21 Aug 2013 15:42:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Aug 2013 15:42:13 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jtaylor@salesforce.com designates 209.85.215.44 as permitted sender) Received: from [209.85.215.44] (HELO mail-la0-f44.google.com) (209.85.215.44) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Aug 2013 15:42:08 +0000 Received: by mail-la0-f44.google.com with SMTP id eo20so491036lab.31 for ; Wed, 21 Aug 2013 08:41:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:references:from:in-reply-to:mime-version:date :message-id:subject:to:content-type; bh=V6Wk0BDaXrpa+ZcHCXt5w9uMnY6WpEwWRmWn+aUJT2A=; b=o1qtxUP7gADnL0jmYANheEfMjiDLHIgah6aojZwnlbFEyBd1LOqywSC2yhB8+YNXjw a8dnrNfAjVEFHAbWIhA+kTQbdOusaUMb4VQn3FaPLXdKfkC2XOH1TXykYByuJoVmfOaq ZrVEgWMgPch1qmJGPO/ZCK9l/nROReiLrQBhmjTOR/yt9tMjtf/lMWwHrvcYh/r1h00Q JGQbO50vnCbWGNgewwy/OpQ5APb4EKpeVRJXuSgr0XiVmRUozNJtkdZ5gyH5BBni3/1i 6sQ7ZNhwpbqztL+YuwRBadRK5gDos/ycu4HCz6YI8oHmZLw0SiX+f49TCzCtvfdYXGu+ WeKQ== X-Gm-Message-State: ALoCoQlOIp2pSbTGQt+LmekzsrkOCTrm9e59RMijS66dmfQ4a/9Hb83ujgmfDD0RR7kQXi8bDK9T X-Received: by 10.112.28.109 with SMTP id a13mr7714780lbh.3.1377099706383; Wed, 21 Aug 2013 08:41:46 -0700 (PDT) References: From: James Taylor In-Reply-To: Mime-Version: 1.0 (1.0) Date: Wed, 21 Aug 2013 08:41:43 -0700 Message-ID: <-2362834569412366241@unknownmsgid> Subject: Re: Does HBase supports parallel table scan if I use MapReduce To: "user@hbase.apache.org" Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org Have a look at Phoenix (https://github.com/forcedotcom/phoenix), a SQL skin over HBase. It does parallel scans and has no map/reduce dependencies. Instead, it compiles your SQL into native HBase calls. Thanks, James @JamesPlusPlus http://phoenix-hbase.blogspot.com On Aug 21, 2013, at 1:08 AM, yonghu wrote: > Thanks. So, to scan the table just using the java program without using > MapReduce will heavily decrease the performance. > > Yong > > > On Tue, Aug 20, 2013 at 6:02 PM, Jeff Kolesky wrote: > >> The scan will be broken up into multiple map tasks, each of which will run >> over a single split of the table (look at TableInputFormat to see how it is >> done). The map tasks will run in parallel. >> >> Jeff >> >> >> On Tue, Aug 20, 2013 at 8:45 AM, yonghu wrote: >> >>> Hello, >>> >>> I know if I use default scan api, HBase scans table in a serial manner, >> as >>> it needs to guarantee the order of the returned tuples. My question is >> if I >>> use MapReduce to read the HBase table, and directly output the results in >>> HDFS, not returned back to client. The HBase scan is still in a serial >>> manner or in this situation it can run a parallel scan. >>> >>> Thanks! >>> >>> Yong >>> >> >> >> >> -- >> *Jeff Kolesky* >> Chief Software Architect >> *Opower* >>