Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of ndimiduk@gmail.com designates
 209.85.220.177 as permitted sender)
MIME-Version: 1.0
From: Nick Dimiduk <ndimiduk@gmail.com>
Date: Mon, 1 Apr 2013 11:00:46 -0700
Message-ID: 
 <CANZa=GuwDXAZUanE8mCDm-AE8ui5mT26HnBs-OwfXURBZOYTQA@mail.gmail.com>
Subject: HBase Types: Explicit Null Support
To: hbase-dev <dev@hbase.apache.org>, hbase-user <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=20cf3079b5eaac542f04d95067cf

--20cf3079b5eaac542f04d95067cf
Content-Type: text/plain; charset=UTF-8

Heya,

Thinking about data types and serialization. I think null support is an
important characteristic for the serialized representations, especially
when considering the compound type. However, doing so in directly
incompatible with fixed-width representations for numerics. For instance,
if we want to have a fixed-width signed long stored on 8-bytes, where do
you put null? float and double types can cheat a little by folding negative
and positive NaN's into a single representation (this isn't strictly
correct!), leaving a place to represent null. In the long example case, the
obvious choice is to reduce MAX_VALUE or increase MIN_VALUE by one. This
will allocate an additional encoding which can be used for null. My
experience working with scientific data, however, makes me wince at the
idea.

The variable-width encodings have it a little easier. There's already
enough going on that it's simpler to make room.

Remember, the final goal is to support order-preserving serialization. This
imposes some limitations on our encoding strategies. For instance, it's not
enough to simply encode null, it really needs to be encoded as 0x00 so as
to sort lexicographically earlier than any other value.

What do you think? Any ideas, experiences, etc?

Thanks,
Nick

--20cf3079b5eaac542f04d95067cf--