Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DD784D728 for ; Wed, 11 Jul 2012 13:59:56 +0000 (UTC) Received: (qmail 86841 invoked by uid 500); 11 Jul 2012 13:59:53 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 86592 invoked by uid 500); 11 Jul 2012 13:59:53 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 86223 invoked by uid 99); 11 Jul 2012 13:59:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jul 2012 13:59:52 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.213.169] (HELO mail-yx0-f169.google.com) (209.85.213.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jul 2012 13:59:44 +0000 Received: by yenr5 with SMTP id r5so1454181yen.14 for ; Wed, 11 Jul 2012 06:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cfotogo.com; s=google; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=q3blys3++EYXiWkP5InZxuEQCO+KqSy9OG9V9I2g6cY=; b=CztaCgpqnL5uBw/7kxSSaOgzesqoYPa/ByoQ4f3vhChaRORHInS8fHkU8f+l+ngFkM ZvmiSMyt9zL/7RhswnZT9Ty63Bn+mdKEvzr8j+vnEppHa6sQsbjF41fFWZusM68X8AkI FU0SdlAM9pOF9Mgawh1Dv+SiNHMuL0aZxEuiA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=q3blys3++EYXiWkP5InZxuEQCO+KqSy9OG9V9I2g6cY=; b=CnrV8o3e+Ug4tNiexvwwKLHHlJ8EoGMtGwwulOG3bk+TgivaWswvTkuREnk8VNygIs yzyPraWRhnCgtmN53TTXcQ+5JmDtu7TrFq3DrONAC56fAAfYRuSTnFQ5AcKGGNwVaTKh MYuXYRKPyNB0NJ5ZCVhgE4/5nHX1APHj5eQukvlpBF5AZLcGm+oulrQv+tTBMtxRV3He YV28cL3qBL4l2HJ0nftQEBgBRkO7FM/wAYlWorp4MN5JEiftO6Amt8qL9Hq1+V+1TE18 aCA89hS9gxM303babtlNEAPTWVmKhUdTIdditmna2Pk85HKt4t3A97O90jQOWtd5e2u0 4Pbg== MIME-Version: 1.0 Received: by 10.66.73.5 with SMTP id h5mr4037558pav.79.1342015162717; Wed, 11 Jul 2012 06:59:22 -0700 (PDT) Received: by 10.68.64.227 with HTTP; Wed, 11 Jul 2012 06:59:22 -0700 (PDT) X-Originating-IP: [98.207.95.222] In-Reply-To: References: Date: Wed, 11 Jul 2012 06:59:22 -0700 Message-ID: Subject: Re: Does anybody use HBase with petabyte sized data ? From: David Schnepper To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=f46d042f9be00e965d04c48e41df X-Gm-Message-State: ALoCoQlBz61e8fX+7T5fu//sJEVemYtUl5rXH2Ig13SYZprCXfZXEfnkHhlWkvPbMaMRcVoNguAg --f46d042f9be00e965d04c48e41df Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable A project we were doing at Yahoo had 2PB of web crawl data on a 1300 node HBase cluster. A presentation we gave at the 2011 Hadoop summit is below http://www.youtube.com/watch?v=3DbyXGqhz2N5M Yahoo has since cancelled the project - but not for any technical reason. Dave Schnepper On Wed, Jul 11, 2012 at 4:41 AM, =EC=9D=B4=EB=B2=94=EC=84=9D wrote: > Hi. > > We are planning to make a big storage to save text and binary data > and considering to use HBase for it. > > Most features of HBase are very appropriate for our program and make easy > to make it. > > But the thing is the capacity of HBase. > ( Nothing I could find about the limit capacity or any experience ... ) > Our data to be stored might be nearly 300TB and still growing bigger. > > If you are working with big-sized HBase or know anyone doing.. > plz let me know it. > > > > > Thanks. > --f46d042f9be00e965d04c48e41df--