synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajika Kumarasiri <raj...@wso2.com>
Subject Re: Fixing SYNAPSE-502
Date Tue, 04 May 2010 04:27:32 GMT
On Mon, May 3, 2010 at 9:45 PM, Hiranya Jayathilaka <hiranya911@gmail.com>wrote:

> Rajika,
>
> On Mon, May 3, 2010 at 5:30 PM, Rajika Kumarasiri <rajika@wso2.com> wrote:
>
>>
>>
>> On Mon, May 3, 2010 at 4:42 PM, Hiranya Jayathilaka <hiranya911@gmail.com
>> > wrote:
>>
>>>
>>>
>>> On Mon, May 3, 2010 at 4:24 PM, Rajika Kumarasiri <rajika@wso2.com>wrote:
>>>
>>>>
>>>>
>>>> On Mon, May 3, 2010 at 4:15 PM, Hiranya Jayathilaka <
>>>> hiranya911@gmail.com> wrote:
>>>>
>>>>> Rajika,
>>>>>
>>>>> On Mon, May 3, 2010 at 3:44 PM, Rajika Kumarasiri <rajika@wso2.com>wrote:
>>>>>
>>>>>> I'd like to hear your feedback before I provide you with a patch
for
>>>>>> the issue[1]. First it's an important improvement to vfs-transport.
>>>>>> I think we can fix the problem by introducing an optional parameter
>>>>>> for the vfs transport , "transport.vfs.TempFileName".
>>>>>
>>>>>
>>>>> Why does the user has to specify this?
>>>>>
>>>> No, it should be an indicator ( a boolean) which suggest to use a temp
>>>> file or not.
>>>>
>>>
>>> Why? Writing to a temp file and then moving the file to the appropriate
>>> place is the safest approach for handling file uploads.
>>>
>>
>> Currently when uploading a file using the vfs transport sender it'll use
>> the provided file name (as a part of the url) as the file name at the ftp
>> server. As you can see this has a problem if one on the ftp server side
>> starts to process the file at the same time since the file is not fully
>> transferred.
>> To avoid this we'll use a temporary file(if the user need a one) at the
>> ftp server for uploading the file, once uploading is completed we can move
>> that temp file to the original one.
>>
>
> IMO this has to be the default behavior. Almost all the time user does not
> have any control over how and when the receiving application reads files
> from the file system. So there is always a chance that the receiving
> application may try to read from a file while Synapse is writing to it. So
> Synapse should always anticipate this scenario and it should always do the
> right thing, which is to write the content to a temp file and then move it
> to the correct location.
>
> For instance we might run into this problem with Synapse itself. Imagine we
> have two Synapse instances. One is writing files to a given directory and
> the other instance is polling on that directory. Without the above described
> approach the second Synapse instance may pick up a half done file.
>
> If you really need to make this configurable, I would still say make this
> the default behavior and provide a way to turn it off if required.
>

I am not sure if we need to make this the default behaviour since this has
some performance implications. Some other similar implementations also use a
temporary file only if one is provided but that's not the default behaviour.
See:
http://help.sap.com/saphelp_nw04/helpdata/en/bc/bb79d6061007419a081e58cbeaaf28/content.htm

Rajika


>
> Thanks,
> Hiranya
>
>
>> Rajika
>>
>>
>>>
>>> Thanks,
>>> Hiranya
>>>
>>>
>>>>
>>>> Rajika
>>>>
>>>>
>>>>> I think this behavior should be there by default in the transport. The
>>>>> transport should figure out suitable names for temp files. Also I don't
>>>>> think we can use a single name for all temp files. There could be situations
>>>>> where multiple worker threads are trying to upload several files to the
same
>>>>> server.
>>>>>
>>>>> Thanks,
>>>>> Hiranya
>>>>>
>>>>>
>>>>>> If  a user has provided this parameter we'll use this file on the
>>>>>> remote server to upload the file. Once the upload is completed we
will move
>>>>>> the temporary file to the original file(which will delete the temp
file).
>>>>>> Comments? Suggestions?
>>>>>>
>>>>>> Rajika
>>>>>>
>>>>>> [1] - https://issues.apache.org/jira/browse/SYNAPSE-502
>>>>>>
>>>>>> --
>>>>>> http://rajikak.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Hiranya Jayathilaka
>>>>> Software Engineer;
>>>>> WSO2 Inc.;  http://wso2.org
>>>>> E-mail: hiranya@wso2.com;  Mobile: +94 77 633 3491
>>>>> Blog: http://techfeast-hiranya.blogspot.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> http://rajikak.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Hiranya Jayathilaka
>>> Software Engineer;
>>> WSO2 Inc.;  http://wso2.org
>>> E-mail: hiranya@wso2.com;  Mobile: +94 77 633 3491
>>> Blog: http://techfeast-hiranya.blogspot.com
>>>
>>
>>
>>
>> --
>> http://rajikak.blogspot.com/
>>
>
>
>
> --
> Hiranya Jayathilaka
> Software Engineer;
> WSO2 Inc.;  http://wso2.org
> E-mail: hiranya@wso2.com;  Mobile: +94 77 633 3491
> Blog: http://techfeast-hiranya.blogspot.com
>



-- 
http://rajikak.blogspot.com/

Mime
View raw message