commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Additions to support Large Linear Regression problems
Date Sat, 25 Jun 2011 16:19:37 GMT
10GB will definitely be fine (and will fit into memory on many modern
machines!).

Somewhere north of a design matrix of 1TB, you will be breaking new ground,
mostly in terms of how well your basic algorithm works.  The LSMR should
still work, but size does matter, often in surprising ways.

On Fri, Jun 24, 2011 at 9:14 PM, Greg Sterijevski <gsterijevski@gmail.com>wrote:

> A large problem would be one where it does not make sense to pull all the
> data into core, whether its 10Gb or 100Tb. While some of these design
> matrices might be sparse, there is no reason to expect it.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message