db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bergquist, Brett" <BBergqu...@canoga.com>
Subject RE: opinions on a less aggressive release of the istats feature?
Date Mon, 14 Mar 2011 12:54:53 GMT
Just from the peanut gallery, this new "istat" feature will be greatly appreciated when available.
 At least from me it will be ;)   It is the one thing missing for a truly "no dba" system.

We have a system installed that provisions and measures network performance for a large wireless
communications vendor that runs 24/7 getting multiple millions of updates per day and having
up to date statistics for the query and reporting features is essential as that data is always
changing.  This system does not have anyone watching it from a dba point of view so it has
to be pretty much self maintaining.

So we are eagerly awaiting this new feature even if it is turned off by default!

Thanks so much for adding this feature.


-----Original Message-----
From: Rick Hillegas [mailto:rick.hillegas@oracle.com] 
Sent: Monday, March 14, 2011 8:45 AM
To: derby-dev@db.apache.org
Subject: Re: opinions on a less aggressive release of the istats feature?

No enthusiasm has been expressed for options (2) or (3). Knut pointed 
out on DERBY-5108 that the shutdown problem also affects existing 
applications which shutdown in the middle of running the 
statistics-updating procedures. I think that istat-on-by-default 
increases the risk that this problem will affect production 
applications. I don't know how to measure that increased risk. I suppose 
it could be significant.

I am going to pursue option (1), i.e., turn of istat by default. I 
propose to do this on the trunk, run regression tests to verify that 
everything looks good, let the nightlies vet the change, then cut the 
branch tomorrow and re-enable istat on the trunk. I believe that turning 
off istat involves the following steps. Does this sound right?

o Back out the changes which enabled the feature 
(derby-4939-1a-enable_istat.diff). This means:

- Default Property.STORAGE_AUTO_INDEX_STATS=false in DataDictionaryImpl.
- Comment out the addition of AutomaticIndexStatisticsTest and 
AutomaticIndexStatisticsMultiTest in the store JUnit suite.

o Regenerate the 10.8.1 release notes:

- Remove the summary line for this feature.
- Change the detailed note for DERBY-4939 so that it says that the 
feature is disabled by default.


On 3/11/11 4:26 PM, Rick Hillegas wrote:
> On 3/11/11 3:59 PM, Mike Matrigali wrote:
>> With the 10.8 release coming up quick and the istats feature put into
>> trunk so recently I was wondering what the community would think about
>> a less aggressive release of the feature.  No matter what I think it
>> should stay enabled by default in trunk.
>> 2 options I can think of are:
>> 1)disable by default, allowing those users that want the feature
>>   to enable it.
> I am comfortable with this default. We can make the change to istat 
> shutdown which you recommend on DERBY-5108 and then continue studying 
> the behavior of this feature on the trunk. It would be disappointing 
> if we had to wait another year before exposing istat-on-by-default. In 
> the long run I think that is a better default for a 0-admin database. 
> I would support a 10.9 release later this year (maybe even in a couple 
> months) when we are more confident about this code.
>> 2)disable by default in all soft upgraded databases.  This means that
>>   applications not upgrading don't get a surprise background task.
> I'm not keen on this approach. If the feature has shutdown-related 
> problems, then that's a showstopper for it being the default in any 
> configuration.
> Another option would be:
> 3) Make the istat shutdown change you recommend and give the code a 
> little while to earn our confidence. I would be willing to delay the 
> release a week for this approach if we think that's enough time to 
> make the change and build confidence. I do not want to push the 
> release out further.
> Thanks,
> -Rick
>> This decision would match other performance/resource decisions 
>> implicitly being made for the system where I think a number of users
>> would benefit with increased resource usage by Derby but think it
>> would be bad to default the system.  These include:
>> o page cache of 1000 can be as small as around 4 meg, which is 
>> incredibly small in the world of 4-8 gig laptops.
>> o sort memory defaults to a very small number currently.
>> o we don't do aggressive background space reclamation which would add
>>   background processing overhead that active apps might not want.

View raw message