Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: neutral (herse.apache.org: local policy)
Message-ID: <457EE454.409@sbcglobal.net>
Date: Tue, 12 Dec 2006 09:18:12 -0800
From: Mike Matrigali <mikem_app@sbcglobal.net>
Reply-To: mikem_app@sbcglobal.net
User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206)
MIME-Version: 1.0
To: derby-dev@db.apache.org
Subject: Re: Question about FetchDescriptor.materialized_cols
References: <x1o3u003u79k.fsf@sun.com>
In-Reply-To: <x1o3u003u79k.fsf@sun.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

as with most "RESOLVES" and "TODO" still in the code, it has been
a long time since anything has happened on them.

FetchDescriptor came about from a profiling/performance pass on the
code.  An array was chosen so that no routine and/or math would
be needed to set/clear the info.  Int's were chosen as likely to be
fasted data structure across all OS/JVM.  Most of this work was done
to optimize various scans (ie. scans of all cols all qualifying, scans
of all cols 1% qualify, scans of some cols all qualifying, scans of some
cols 1% qualifying).  At the time this was the
fastest data structure, faster than bits - and especially faster
than bits if number of bits might be large (ie. bigger than 8 or 32 -
so that some array reference was necessary).

I haven't thought about what it would mean to cache the offset, one 
would need to have some sort of validation of the value after the
latch were released.  The
case where one needs to find the column again after qualifying (where
this code is), is a subsequent update of a column after this row
is returned to the caller in a scan as a qualifying row.  Note that
the "normal" qualifying path for many apps is find a by key in index
and then update non-key field of heap - in that case this code is
not used.

The more interesting issue is the general cost of getting to a column,
let me start a separate thread on that.

Dyre.Tjeldvoll@Sun.COM wrote:
> Today this is an int[] but it appears to be used as a boolean. A
> possible explanation for this can be found in a comment in
> StoredPage.java:
> 
>                 // RESOLVE (mikem) - right now value of entry is useless, it
>                 // is an int so that in the future we could cache the offset
>                 // to fields to improve performance of getting to a column 
>                 // after qualifying.
>                 materializedCols[col_id] = offset_to_row_data;
> 
> Can someone (mikem?) explain a bit more about this optimization and
> what it would take to implement it? What would the potential gains be?
> Is it a JIRA for this?
>