apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Weise <tho...@datatorrent.com>
Subject Re: Vagrant folders under Malhar/lib
Date Thu, 03 Sep 2015 06:17:42 GMT
Chandni,

Agreed. See whether the tests work with the synchronous storage agent. If
yes, change them. The default needs to remain async.

Thomas


On Wed, Sep 2, 2015 at 11:05 PM, Chandni Singh <chandni@datatorrent.com>
wrote:

> Hi,
>
> I would like to know what was the reason to use AsyncFSStorageAgent with
> StramLocalCluster?
> StramLocalCluster is mainly for testing in a non-distributed mode and I am
> unclear how AsyncFSStorageAgent is helpful in this mode.
>
> Thanks,
> Chandni
>
> On Wed, Sep 2, 2015 at 10:45 PM, Chandni Singh <chandni@datatorrent.com>
> wrote:
>
>> This is because of recent changes to StramLocalCluster where
>> AsyncFSStorageAgent is used for checkpointing
>>
>> dag.setAttribute(OperatorContext.STORAGE_AGENT, new AsyncFSStorageAgent(new Path(pathUri,
LogicalPlan.SUBDIR_CHECKPOINTS).toString(), null));
>>
>> The AsyncFSStorageAgent(String path, Configuration conf) uses "." as localBasePath
and therefore creates sub-directories per operator in the current working directory.
>>
>> I am going to create a ticket to address this and will fix it.
>>
>> -Chandni
>>
>>
>> On Wed, Sep 2, 2015 at 7:13 PM, Chandni Singh <chandni@datatorrent.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I can see empty folders getting created under Malhar/lib called '1' and
>>> '2'.
>>> I think this is because of using LocalMode to run a test application.
>>>
>>>
>>> If anyone has checked in such cases please do check and let us know.
>>>
>>> Thanks,
>>> Chandni
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message