Mailing-List: contact dev-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of tsaloranta@gmail.com designates
 209.85.210.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <589876BA-E16B-4477-A329-5549AC5648F0@morningstar.com>
References: 
 <CAH3cagMWrgptNtBLSQLrpmJFZFhoN64eqg5iaBp5Eu6=_Sx9aA@mail.gmail.com>
	<589876BA-E16B-4477-A329-5549AC5648F0@morningstar.com>
Date: Wed, 28 Mar 2012 19:54:13 -0700
Message-ID: 
 <CAGrxA24==zuzhZZeM+6Tyc1anqJa62THMoaT7Qb2=dMQV7DqGw@mail.gmail.com>
Subject: Re: Document storage
From: Tatu Saloranta <tsaloranta@gmail.com>
To: dev@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Wed, Mar 28, 2012 at 6:59 PM, Jeremiah Jordan
<JEREMIAH.JORDAN@morningstar.com> wrote:
> Sounds interesting to me. =A0I looked into adding protocol buffer support=
 at one point, and it didn't look like it would be too much work. =A0The tr=
icky part was I also wanted to add indexing support for attributes of the i=
nserted protocol buffers. =A0That looked a little trickier, but still not i=
mpossible. =A0Though other stuff came up and I never got around to actually=
 writing any code.
> JSON support would be nice, especially if you figured out how to get buil=
t in indexing of the attributes inside the JSON to work =3D).

Also, for whatever it's worth, it should be trivial to add support for
Smile (binary JSON serialization):
http://wiki.fasterxml.com/SmileFormatSpec
since its logical data structure is pure JSON, no extensions or
subsetting. The main Java impl is by Jackson project, but there is
also a C codec (https://github.com/pierre/libsmile), and prototypes
for PHP and Ruby bindings as well.
But for all data it's bit faster, bit more compact; about 30% for
individual items, but more (40 - 70%) for data sequences (due to
optional back-referencing).

JSON and Smile can be auto-detected from first 4 bytes or so, reliably
and efficiently, so one should be able to add this either
transparently or explicitly.
One could even transcode things on the fly -- store as Smile, expose
filtered results as JSON (and accept JSON or both). This could reduce
storage cost while keep the benefits of flexible data format.

-+ Tatu +-