Mailing-List: contact commons-dev-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Jakarta Commons Developers List" <commons-dev@jakarta.apache.org>
Message-ID: <20030606172112.8624.qmail@web41703.mail.yahoo.com>
Date: Fri, 6 Jun 2003 10:21:12 -0700 (PDT)
From: Phil Steitz <steitzp@yahoo.com>
Subject: [math] proposed ordering for task list, scope of initial release
To: commons-dev@jakarta.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Here is a *proposed* ordering for the task list, with a little commentary
added.

One thing that I want to make *very* clear up front, is that I *never* intended
the task list or the items listed in the scope section of the proposal to be
definitive.  All that is definitive are the guiding principles, which just try
to keep us focused on stuff that people will find both useful and easy to use. 
I expected that the actual contents of the first release would include some
things not on the list and would exclude some of the things there.  At this
stage, as Jouzas pointed out, it is more important for us to build community
than to rush a release out the door. So if there are things that fit the
guidelines that others would like to contribute, but which are not on the list,
*please* suggest them.  Also, for those who may not have dug into the code, but
who may be interested in contributing, please rest assured that deep
mathematical knowledge is not required to help. We can review implementations
and deal with mathematical problems as they arise, using our small but growing
community as a resource.  The same is obviously true on the the Java/OS tools
side -- no need to be an expert to contribute.

OK, long-winded disclaimer aside, here is how I see the task list ordered:

* The RealMatrixImpl class is missing some key method implementations. The
critical thing is solution of linear systems. We need to implement a
numerically sound solution algorithm. This will enable inverse() and also
support general linear regression. -- I think that Brent is working on this.   

* Improve numerical accuracy of Univariate and BivariateRegression statistical
computations. Encapsulate basic double[] |-> double mean, variance, min, max
computations using improved formulas and add these to MathUtils. (probably
should add float[], int[], long[] versions as well.) Then refactor all
univariate implementations that use stored values (including UnivariateImpl
with finite window) to use the improved versions. -- Mark?  I am chasing down
the TAS reference to document the source of the _NR_ formula, which I will add
to the docs if someone else does the implementation.

* Define full package structure and develop user's guide following the package
structure.  I have started work on the user's guide, but found this impossible
without the package structure defined.  I will post a separate message
summarizing what has been proposed up to now and making a recommendation.

* t-test statistic needs to be added and we should probably add the capability
of actually performing t- and chi-square tests at fixed significance levels
(.1, .05, .01, .001). -- This is virtually done, just need to define a nice,
convenient interface for doing one- and two-tailed tests.  Thanks to Brent, we
can actually support user-supplied significance levels (next item)

* numerical approximation of the t- and chi-square distributions to enable
user-supplied significance levels.  See above.  Someone just needs to put a
fork in this. Tim? Brent?

* *new* add support for F distribution and F test, so that we can report
signinficance level of correlation coefficient in bivariate regression /
signinficance of model.  I will do this if no one else wants to.

* Framework and implementation strategie(s) for finding roots or real-valued
functions of one (real) variable.  Here again -- largely done.  I would prefer
to wait until J gets back and let him submit his framework and R. Brent's
algorithm.  Then "our" Brent's implementation and usage can be integrated
(actually not much to do, from the looks of the current code) and I will add my
"bean equations" stuff (in progress).

* Extend distribution framework to support discrete distributions and implement
binomial and hypergeometric distributions.  I will do this if no one else wants
to.  If someone else does it, you should make sure to use the log binomials in
computations.

* Exponential growth and decay (set up for financial applications) I think this
is just going to be a matter of finding the right formulas to add to MathUtils.
 I don't want to get carried away with financial computations, but some simple,
commonly used formulas would be a nice addition to the package. We should also
be thinking about other things to add to MathUtils -- religiously adhering to
th guiding principles, of course.  Al's sign() is an excellent example of the
kind of thing that we should be adding, IMHO.

* Polynomial Interpolation -- let Al tell us what to do here.  Even better, let
Al do it (he he).   

* Javadoc generation currently throws 35 warnings. Bring the Javadoc into
compliance (i.e. reach zero warnings). A community effort :-)

* Clover tests show lots of gaps in test path coverage. Get all tests to 100%
coverage

* CheckStyle with modified properties still shows many errors. Clean these up.

* ComplexNumber implementation, supporting C9x Annex G spec.  I would prefer to
wait on this, but I could be persuaded to finish my implementation (or better,
include someone else's fully compliant implementation) if others feel strongly.

* Addition of a Arithmetic Geometric Mean.  I would wait in this as well.
---------------------------------------------

My proposal for the first release is that we

1) complete the tasks above, other than the last two

2) step back and take a critical look at the design decisions that we have made
up to now and consider refactoring some things (Univariate world, RealMatrix,
and TestStatistic, esp.) Also obviously finalize package structure.  This task
obviously needs to run concurrently with the ones above.

3) develop the user's guide and validate all algorithms and documentation.

4) do some performance testing and as much numerical stability validation as we
can.   

5) follow the Jakarta sandbox process for alpha release

Phil


__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org