Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9F455F2E4 for ; Thu, 13 Nov 2014 05:55:09 +0000 (UTC) Received: (qmail 27035 invoked by uid 500); 13 Nov 2014 05:55:07 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 26962 invoked by uid 500); 13 Nov 2014 05:55:07 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 26950 invoked by uid 99); 13 Nov 2014 05:55:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Nov 2014 05:55:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jianshi.huang@gmail.com designates 209.85.217.177 as permitted sender) Received: from [209.85.217.177] (HELO mail-lb0-f177.google.com) (209.85.217.177) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Nov 2014 05:54:41 +0000 Received: by mail-lb0-f177.google.com with SMTP id z12so3712084lbi.8 for ; Wed, 12 Nov 2014 21:53:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=IHLDQE402sYtPZThgatq7ilYeakJ6fPLybhPqkq7n+A=; b=xONjOy1LzLRz1lTgcsnZAha7yVWIvS8lASIZXQX43+cOVGIZV12cZTNDPn8N2r06tT +lyaoSgT8zcegEeUKhQksjpce3jHCdHJ/ua2ca6k+fLgoVuv5bKbjZ/+mKQppIGB1Gbd p6lRjYtBxs09jKssm4tsHZfcbsHWxkP2OLGLslU8MgrZE1eqIBUzpAc8Af4f1BFbMC5p cyb1FxA1HHlNxRdWT/w+GfDGfhQEeoG0ko3Xz7OuXDNPE5PBM3+/sPCTpFLt0bd/IQ9F JlGakSwhPe+7raYxfcSn+QoqFq3bezzyYoLoXYdVfB+n0gQpgaBrM2cNRCS9GNmH+qyh AEig== X-Received: by 10.113.5.7 with SMTP id ci7mr299991lbd.9.1415858035971; Wed, 12 Nov 2014 21:53:55 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.12.200 with HTTP; Wed, 12 Nov 2014 21:53:35 -0800 (PST) In-Reply-To: <6AB6526B-AFA8-49F3-8CCA-64496B4E373B@gmail.com> References: <6AB6526B-AFA8-49F3-8CCA-64496B4E373B@gmail.com> From: Jianshi Huang Date: Thu, 13 Nov 2014 13:53:35 +0800 Message-ID: Subject: Re: Storing JSON in HBase value cell, which serialization format is most compact? To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=001a1133a890495fbd0507b723b5 X-Virus-Checked: Checked by ClamAV on apache.org --001a1133a890495fbd0507b723b5 Content-Type: text/plain; charset=UTF-8 I thought FASTDIFF was only for rowkey and columns, great if it also works in value cell. And thanks for the bjson link! Jianshi On Thu, Nov 13, 2014 at 1:18 PM, Ted Yu wrote: > There is FASTDIFF data block encoding. > > See also http://bjson.org/ > > Cheers > > On Nov 12, 2014, at 9:08 PM, Jianshi Huang > wrote: > > > Hi, > > > > I'm currently saving JSON in pure String format in the value cell and > > depends on HBase' block compression to reduce the overhead of JSON. > > > > I'm wondering if there's a more space efficient way to store JSON? > > (there're lots of 0s and 1s, JSON String actually is an OK format) > > > > I want to keep the value as a Map since the schema of source data might > > change over time. > > > > Also is there a DIFF based encoding for values? Since I'm storing > > historical data (snapshot data) and changes between adjacent value cells > > are relatively small. > > > > > > Thanks, > > -- > > Jianshi Huang > > > > LinkedIn: jianshi > > Twitter: @jshuang > > Github & Blog: http://huangjs.github.com/ > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/ --001a1133a890495fbd0507b723b5--