reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhruv Mahajan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (REEF-1224) IMRU Fault Tolerance - Separate Data downloading from Task injection
Date Tue, 22 Mar 2016 19:04:25 GMT

    [ https://issues.apache.org/jira/browse/REEF-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207052#comment-15207052
] 

Dhruv Mahajan edited comment on REEF-1224 at 3/22/16 7:04 PM:
--------------------------------------------------------------

I just checked the way code was running. 
In Service configuration, I passed on the following configuration:

{code}
var dataLoadingContext =
                TangFactory.GetTang()
                    .NewConfigurationBuilder()
                    .BindSetEntry<ContextConfigurationOptions.StartHandlers, DataLoadingContext<TDataHandler>,
IObserver<IContextStart>>(GenericType<ContextConfigurationOptions.StartHandlers>.Class,
                            GenericType<DataLoadingContext<TDataHandler>>.Class)
                    .Build();

            var serviceConf =
                TangFactory.GetTang()
                    .NewConfigurationBuilder(ServiceConfiguration.ConfigurationModule.Build(),
                        dataLoadingContext,
                        partitionDescriptor.GetPartitionConfiguration())
                    .Build();
{code}
Below is the {{DataLoadingContext}} Class

{code}
    internal class DataLoadingContext<T> : IObserver<IContextStart>
    {
        private static readonly Logger Logger = Logger.GetLogger(typeof(DataLoadingContext<T>));

        [Inject]
        internal DataLoadingContext(IInputPartition<T> partition)
        {
            Console.WriteLine("ENTERED CONTEXT");
        }

        public void OnNext(IContextStart value)
        {
        }

        public void OnError(Exception error)
        {
            Exceptions.Throw(error, "Error occured in Data Loading context start", Logger);
        }

        public void OnCompleted()
        {
        }
    }
{code}

Now when I build GroupComm Service on top of it and SubmitContext and everything, the above
context gets called again as expected. However, it seems that {{DataLoadingContext}} constructor
and {{IInputPartition}} constructors are also being called again. I do not think this is an
expected behavior right?

Am I doing something wrong?


was (Author: dkm2110):
I just checked the way code was running. 
In Service configuration, I passed on the following configuration:

{code}
var dataLoadingContext =
                TangFactory.GetTang()
                    .NewConfigurationBuilder()
                    .BindSetEntry<ContextConfigurationOptions.StartHandlers, DataLoadingContext<TDataHandler>,
IObserver<IContextStart>>(GenericType<ContextConfigurationOptions.StartHandlers>.Class,
                            GenericType<DataLoadingContext<TDataHandler>>.Class)
                    .Build();

            var serviceConf =
                TangFactory.GetTang()
                    .NewConfigurationBuilder(ServiceConfiguration.ConfigurationModule.Build(),
                        dataLoadingContext,
                        partitionDescriptor.GetPartitionConfiguration())
                    .Build();
{code}
Below is the {{DataLoadingContext}} Class

{code}
    internal class DataLoadingContext<T> : IObserver<IContextStart>
    {
        private static readonly Logger Logger = Logger.GetLogger(typeof(DataLoadingContext<T>));

        [Inject]
        internal DataLoadingContext(IInputPartition<T> partition)
        {
            Console.WriteLine("ENTERED CONTEXT");
        }

        public void OnNext(IContextStart value)
        {
        }

        public void OnError(Exception error)
        {
            Exceptions.Throw(error, "Error occured in Data Loading context start", Logger);
        }

        public void OnCompleted()
        {
        }
    }
{code}

Now when I build GroupComm Service on top of it and SubmitContext and everything, the above
context gets called again as expected. However, it seems that DataLoadingContext constructor
and IInputPartition cosntructors are also being called again. I do not think this is an expected
behaviour right?

Am I doing something wrong?

> IMRU Fault Tolerance - Separate Data downloading from Task injection
> --------------------------------------------------------------------
>
>                 Key: REEF-1224
>                 URL: https://issues.apache.org/jira/browse/REEF-1224
>             Project: REEF
>          Issue Type: Improvement
>          Components: IMRU, REEF.NET
>            Reporter: Julia
>            Assignee: Dhruv Mahajan
>
> Currently in IMRU, data downloading happens during the Task injection. It couples the
data and Task object. In Fault tolerant case, we would like to only resubmit a task but use
the data that have been downloaded, That requires us to decouple those two portions. For example,
data downloading portion can be attached to Context, and we can then resubmit a task on the
same context. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message