reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Chung (JIRA)" <>
Subject [jira] [Updated] (REEF-1412) Use a two-file approach for DFSEvaluatorLogOverwriteWriter
Date Thu, 02 Jun 2016 18:49:59 GMT


Andrew Chung updated REEF-1412:
    Summary: Use a two-file approach for DFSEvaluatorLogOverwriteWriter  (was: Improve efficiency
of DFSEvaluatorLogOverwriteWriter)

> Use a two-file approach for DFSEvaluatorLogOverwriteWriter
> ----------------------------------------------------------
>                 Key: REEF-1412
>                 URL:
>             Project: REEF
>          Issue Type: Sub-task
>          Components: REEF Driver, REEF.NET Driver
>            Reporter: Andrew Chung
>            Assignee: Andrew Chung
> {{DFSEvaluatorLogOverwriteWriter}} currently reads from original EvalutorChangesLog before
overwriting it. This can become a problem when the job has been running for a long time. We
can make this more efficient by simply recording the Evaluators it expects in a log.
> Another problem that needs to be addressed is that it currently deletes the old EvaluatorChangesLog
prior to writing the new one. Instead, we should use a two-file approach, where an alternative
file, EvaluatorChangesLog.alt, is used in tandem with EvaluatorChangesLog.
> On read, the newest between EvaluatorChangesLog and EvaluatorChangesLog.alt should always
be read. 
> On write, we always overwrite the older file between EvaluatorChangesLog and EvaluatorChangesLog.alt.

This message was sent by Atlassian JIRA

View raw message