reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Weimer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-1842) Making IMRU task and input data association deterministic
Date Thu, 27 Jul 2017 01:19:00 GMT

    [ https://issues.apache.org/jira/browse/REEF-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102552#comment-16102552
] 

Markus Weimer commented on REEF-1842:
-------------------------------------

Are you envisioning something where the user supplies the mapping? Or more like a hash based
approach which is random, but deterministic?

> Making IMRU task and input data association deterministic
> ---------------------------------------------------------
>
>                 Key: REEF-1842
>                 URL: https://issues.apache.org/jira/browse/REEF-1842
>             Project: REEF
>          Issue Type: Task
>            Reporter: Julia
>            Assignee: Julia
>            Priority: Critical
>
> Currently, the tasks added to the typology on communication group is based on the sequence
of contexts received. While the input data assigned to context can be random especially when
some evaluators fail and new evaluators/contexts are requested. This makes the training result
undeterministic given same input data set. 
> To make the result deterministic for given set of input data, we would like to make tasks
in the typology keep the same order with the same input data file associated. To achieve it,
the mapping between the context id and partitionDescriptor id should be fixed once assigned.
And when adding tasks to the communication group, the order should be based on the sorted
id to ensure it is deterministic. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message