camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schneider <ch...@die-schneider.net>
Subject Re: Zip format problem
Date Mon, 05 Oct 2009 22:27:53 GMT
If just a normal move would be used it could still happen that the 
second route starts while half of the files have been moved. I think it 
could work if the files are moved in the sort order. If this moving is 
done directly after the unpacking it should work. I have no idea though 
if this could be expressed in Camel. On the other hand including this 
move logic in uncompress is probably not such a good idea.

Greetings

Christian


Hadrian Zbarcea schrieb:
> What about unpack in a temp dir and after unpacking is complete mv the 
> temp dir to the destination (tempCacheFolder in this case)?
>
> My $0.02,
> Hadrian
>
>
> On Oct 5, 2009, at 6:03 PM, Christian Schneider wrote:
>
>> Hi Vladimir,
>>
>> looks almost good. The problem is though that the second route 
>> already starts processing while the first route still unpacks the 
>> zip. We had to use a special marker file and a special
>> filter for the files to make sure it waits till the zip is fully 
>> unpacked. Our solution was quite special though.
>>
>> I think a general solution could be to somehow block the second route 
>> while the first route is active. I have no idea though if this can be 
>> done with camel. Any ideas?
>>
>> Greetings
>>
>> Christian
>>
>> Vladimir Okhotnikov schrieb:
>>> After having slept on it, I think that since you will have to resort 
>>> to use
>>> disk as a kind of cache anyway in general case, you can just as well do
>>>
>>> from(...).uncompress(Zip).to("file://tempCacheFolder?...");
>>> from("file://tempCacheFolder?sort=${file:name}").process(...
>>>
>>> Which means it is actually not worth it to create more elaborate 
>>> solution
>>> for extracting files from zip archives in particular order, given the
>>> considerations that it a) would not work without penalty on solid 
>>> archives
>>> and b) is probably kind of rare requirement - in some cases sorting of
>>> unarchived files is irrelevant, in other it is possible to reorder
>>> processing results before aggregation instead.
>>>
>>> What do you think?
>>>
>>>
>>> Christian Schneider wrote:
>>>
>>>> Vladimir Okhotnikov schrieb:
>>>>
>>>>> Christian Schneider wrote:
>>>>>
>>>>>> Btw. In our scenario we had the requirement that the files from 
>>>>>> the zip had to be processed in a certain order. In our case the 
>>>>>> processing should be done in order of the filenames.
>>>>>> Any idea how this could be expressed?
>>>>>>
>>>>>>
>>>>> decompress(Zip).resequencer()... ?
>>>>>
>>>> Logically this would work but I fear it could consume much memory. 
>>>> Resequencer canĀ“t be made streaming but perhaps it can use a disk.
>>>>
>>>> Greetings
>>>>
>>>> Christian
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>>
>>>> Christian Schneider
>>>> ---
>>>> http://www.liquid-reality.de
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> -- 
>>
>> Christian Schneider
>> ---
>> http://www.liquid-reality.de
>>
>
>


-- 

Christian Schneider
---
http://www.liquid-reality.de


Mime
View raw message