Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3FF34D59F for ; Thu, 10 Jan 2013 19:25:34 +0000 (UTC) Received: (qmail 39179 invoked by uid 500); 10 Jan 2013 19:25:32 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 39100 invoked by uid 500); 10 Jan 2013 19:25:32 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 39092 invoked by uid 99); 10 Jan 2013 19:25:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 19:25:32 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dontariq@gmail.com designates 209.85.220.169 as permitted sender) Received: from [209.85.220.169] (HELO mail-vc0-f169.google.com) (209.85.220.169) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 19:25:27 +0000 Received: by mail-vc0-f169.google.com with SMTP id gb23so714022vcb.0 for ; Thu, 10 Jan 2013 11:25:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=LJcJ9r2CjOze20tHFopzKBiT+BzLob8ThW53CbOSXtQ=; b=FDYI0MPWT0/XH/4f62OHG5WSs6Mho6H4ik+28oXySuz1N+MczL2p9HVB4QaGeTyVPw aWLTai7MmtjLfJIU3DOVAPyJerEhCokuCDxjJe3uhUxKFDhTQj+4bzl/DxGjhxooUzzu D1IZw0E2hfoeZPYuHnGFROA81238qwRmJvXQK/5bgCk9V8BcMc0EmNGhnb+NLhjI3Y+T ewGhXxJLqVu0D7WaOIs8WHlLkS6LWvBTuJUmfAoIyaCoPtzP8HEqgrq1ajp196jYpDko OWgvEnp4HzBd5pwmOrcfGn2raXd6e/lO8/LCg1JnjqRNdW3L4Nz6gJlBhsA4mOyfT4pI 3/jA== Received: by 10.58.64.51 with SMTP id l19mr95220202ves.15.1357845906469; Thu, 10 Jan 2013 11:25:06 -0800 (PST) MIME-Version: 1.0 Received: by 10.58.34.16 with HTTP; Thu, 10 Jan 2013 11:24:26 -0800 (PST) In-Reply-To: <8821C89B-F82C-40CB-9C39-276CADDF55E5@hortonworks.com> References: <1357380684848-4036184.post@n3.nabble.com> <3702994083872207037@unknownmsgid> <8821C89B-F82C-40CB-9C39-276CADDF55E5@hortonworks.com> From: Mohammad Tariq Date: Fri, 11 Jan 2013 00:54:26 +0530 Message-ID: Subject: Re: Storing images in Hbase To: "user@hbase.apache.org" Content-Type: multipart/alternative; boundary=047d7b6d8102ea33d804d2f422b7 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b6d8102ea33d804d2f422b7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Jack, Leonid, I request you guys to please continue the discussion through the thread itself if possible for you both. I would like to know about Jack's setup. I too find it quite interesting. Many thanks. Warm Regards, Tariq https://mtariq.jux.com/ On Fri, Jan 11, 2013 at 12:50 AM, Leonid Fedotov wrote: > Jack, > yes, this is very interesting to know your setup details. > Could you please provide more information? > Or we can take this off the list if you like=85 > > Thank you! > > Sincerely, > Leonid Fedotov > > On Jan 10, 2013, at 9:24 AM, Jack Levin wrote: > > > We stored about 1 billion images into hbase with file size up to 10MB. > > Its been running for close to 2 years without issues and serves > > delivery of images for Yfrog and ImageShack. If you have any > > questions about the setup, I would be glad to answer them. > > > > -Jack > > > > On Sun, Jan 6, 2013 at 1:09 PM, Mohit Anchlia > wrote: > >> I have done extensive testing and have found that blobs don't belong i= n > the > >> databases but are rather best left out on the file system. Andrew > outlined > >> issues that you'll face and not to mention IO issues when compaction > occurs > >> over large files. > >> > >> On Sun, Jan 6, 2013 at 12:52 PM, Andrew Purtell > wrote: > >> > >>> I meant this to say "a few really large values" > >>> > >>> On Sun, Jan 6, 2013 at 12:49 PM, Andrew Purtell > >>> wrote: > >>> > >>>> Consider if the split threshold is 2 GB but your one row contains 10 > GB > >>> as > >>>> really large value. > >>> > >>> > >>> > >>> > >>> -- > >>> Best regards, > >>> > >>> - Andy > >>> > >>> Problems worthy of attack prove their worth by hitting back. - Piet > Hein > >>> (via Tom White) > >>> > > --047d7b6d8102ea33d804d2f422b7--