camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Okhotnikov <vokhotni...@gmail.com>
Subject Re: Zip format problem
Date Tue, 06 Oct 2009 10:21:42 GMT

And also, there is a high probability, that if both temporary and source
folders are in the same filesystem, the mv would just rename the temporary
folder, not move each file, so the Hadrian's suggestion would actually work
pretty well.


Christian Schneider wrote:
> 
> If just a normal move would be used it could still happen that the 
> second route starts while half of the files have been moved. I think it 
> could work if the files are moved in the sort order. If this moving is 
> done directly after the unpacking it should work. I have no idea though 
> if this could be expressed in Camel. On the other hand including this 
> move logic in uncompress is probably not such a good idea.
> 
> Greetings
> 
> Christian
> 
> 
> Hadrian Zbarcea schrieb:
>> What about unpack in a temp dir and after unpacking is complete mv the 
>> temp dir to the destination (tempCacheFolder in this case)?
>>
>> My $0.02,
>> Hadrian
>>
>>
>> On Oct 5, 2009, at 6:03 PM, Christian Schneider wrote:
>>
>>> Hi Vladimir,
>>>
>>> looks almost good. The problem is though that the second route 
>>> already starts processing while the first route still unpacks the 
>>> zip. We had to use a special marker file and a special
>>> filter for the files to make sure it waits till the zip is fully 
>>> unpacked. Our solution was quite special though.
>>>
>>> I think a general solution could be to somehow block the second route 
>>> while the first route is active. I have no idea though if this can be 
>>> done with camel. Any ideas?
>>>
>>> Greetings
>>>
>>> Christian
>>>
>>> Vladimir Okhotnikov schrieb:
>>>> After having slept on it, I think that since you will have to resort 
>>>> to use
>>>> disk as a kind of cache anyway in general case, you can just as well do
>>>>
>>>> from(...).uncompress(Zip).to("file://tempCacheFolder?...");
>>>> from("file://tempCacheFolder?sort=${file:name}").process(...
>>>>
>>>> Which means it is actually not worth it to create more elaborate 
>>>> solution
>>>> for extracting files from zip archives in particular order, given the
>>>> considerations that it a) would not work without penalty on solid 
>>>> archives
>>>> and b) is probably kind of rare requirement - in some cases sorting of
>>>> unarchived files is irrelevant, in other it is possible to reorder
>>>> processing results before aggregation instead.
>>>>
>>>> What do you think?
>>>>
>>>>
>>>> Christian Schneider wrote:
>>>>
>>>>> Vladimir Okhotnikov schrieb:
>>>>>
>>>>>> Christian Schneider wrote:
>>>>>>
>>>>>>> Btw. In our scenario we had the requirement that the files from

>>>>>>> the zip had to be processed in a certain order. In our case the

>>>>>>> processing should be done in order of the filenames.
>>>>>>> Any idea how this could be expressed?
>>>>>>>
>>>>>>>
>>>>>> decompress(Zip).resequencer()... ?
>>>>>>
>>>>> Logically this would work but I fear it could consume much memory. 
>>>>> Resequencer canĀ“t be made streaming but perhaps it can use a disk.
>>>>>
>>>>> Greetings
>>>>>
>>>>> Christian
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -- 
>>>>>
>>>>> Christian Schneider
>>>>> ---
>>>>> http://www.liquid-reality.de
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> -- 
>>>
>>> Christian Schneider
>>> ---
>>> http://www.liquid-reality.de
>>>
>>
>>
> 
> 
> -- 
> 
> Christian Schneider
> ---
> http://www.liquid-reality.de
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Zip-format-problem-tp25723682p25766202.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message