mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jake Mannix (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAHOUT-319) SVD solvers should be gracefully stoppable/restartable
Date Tue, 19 Apr 2011 15:42:05 GMT

     [ https://issues.apache.org/jira/browse/MAHOUT-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jake Mannix updated MAHOUT-319:
-------------------------------

    Attachment: MAHOUT-319.diff

Updated patch, which adds HDFS persistence to the LanczosState, with unit/integration test.

Integrated API changes with EigencutsDriver and SpectralKMeansDriver (but not the HDFS persistence
piece).

Still need to integrate HDFS persistence with the DistributedLanczosDriver before this bug
is finished off.

> SVD solvers should be gracefully stoppable/restartable
> ------------------------------------------------------
>
>                 Key: MAHOUT-319
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-319
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>    Affects Versions: 0.3
>            Reporter: Jake Mannix
>            Assignee: Jake Mannix
>             Fix For: 0.5
>
>         Attachments: MAHOUT-319.diff, MAHOUT-319.patch
>
>
> LanczosSolver, DistributedLanczosSolver, and HebbianSolver all keep copious amounts of
memory-resident data which is lost if the app crashes or is killed (OOM, forgetting to run
in a screen session, and losing net connectivity to the server running it, etc...).  
> These algorithms (and many other Mahout processes!) should enable a pluggable "persist
state" mechanism (to HDFS, RDBMS, local disk, key-value store, etc), and similarly, a way
to pick up and start from such a state.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message