camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: NPE for FTP endpoint
Date Tue, 03 Jan 2012 16:04:34 GMT
Hi

Take a look at the polling consumer EIP pattern
http://camel.apache.org/polling-consumer.html

It allows you to poll messages on-demand from any Camel endpoint.

And you can use the ConsumerTemplate API to do it from a few lines of code.

Then you do not need any Camel routing, etc.



On Tue, Jan 3, 2012 at 3:48 PM, David Wynter <david_wynter@yahoo.com> wrote:
> Hi,
>
> I implemented my HaltProcessor. But it still does not behave as I thought. I could not
see where in the documentation it explains the route behaviour for shutdown. I have this:
>
>         String sourcePath = createFromUri();
>         if(sourcePath == null)
>         return;
>         String to = localName.length()>0?localName:"";
>         RouteDefinition routeD = new RouteDefinition();
>               routeD.from(sourcePath);
>
>         routeD.to(localPath+"?fileExist=Override");
>         HaltProcessor pHalt = new HaltProcessor();
>         pHalt.setContext(camelContext);
>         routeD.routeId(sourcePath);
>         pHalt.setRouteId(sourcePath);
>         routeD.process(pHalt);
>            camelContext.addRouteDefinition(routeD);
>
>                 camelContext.startRoute(routeD);
>
> where Halt Processor has this process method body:
>
> if(context!=null && routeId != null) {
>         // remove myself from the in flight registry so we can stop this route without
trouble
>         context.getInflightRepository().remove(exchange);
>         // stop this route
>         context.stopRoute(routeId);
>
> My problem is this. The startRoute occurs and starts the route then immediately hits
the HaltProcessor. It hits the HaltProcessor before completing the ftp download, which might
be a 2KB file or a hundred 1MB files. Then it all exits and my next task in a workflow executes.
It depends on the file being completely downloaded from the download the route provides. But
it seems the calling thread is terminated but some other thread continues the task of doing
the download, this isn't completed by the time the route returns.
>
> My requirement is simple, download the file from ftp, do not return until this is done,
do it once only, do not poll, just return at this point. Do I need my own ShutdownStrategy
for this simple requirement, if so, is there a listener I can setup to know the file has completed
the download?
>
> A separate problem is even using noop=true it still polls.
>
> Thx.
>
> David
>
>
>
> ________________________________
>  From: Taariq Levack <taariql@gmail.com>
> To: "users@camel.apache.org" <users@camel.apache.org>
> Sent: Thursday, 29 December 2011, 19:35
> Subject: Re: NPE for FTP endpoint
>
> The processor[1] is exactly that class so give it a try, also see the file language [2]
for dynamically setting filename.
>
> [1] http://camel.apache.org/processor.html
> [2] http://camel.apache.org/file-language.html
>
> On 29 Dec 2011, at 9:14 PM, David Wynter <david_wynter@yahoo.com> wrote:
>
>> Hi,
>>
>> Yes it is a dynamic path for producer and consumer. Restricted in this case as ftp
to file or in another case from file to ftp. This use case is very restricted, my intent is
to use Camel in a less restricted way in another use case from within this same application
later, otherwise I would have just used FTPClient
>>
>> So you are saying I just need a class in the route that has access to the exchange
to be able to stop it. I'll have a dig around in the JavaDoc to see what I can find. Are there
more examples somewhere of how to use the API?
>>
>> Thx.
>>
>>
>> ________________________________
>> From: Taariq Levack <taariql@gmail.com>
>> To: "users@camel.apache.org" <users@camel.apache.org>
>> Sent: Thursday, 29 December 2011, 18:56
>> Subject: Re: NPE for FTP endpoint
>>
>> Why do you want to create a processor definition that way?
>> The example only uses the processor to illustrate the point, it has access to the
exchange and can easily remove it from the inflight repo of the camel context.
>> To do it your way I don't know without an IDE handy, I avoid working with low level
camel API in my apps, so maybe first try to avoid it in this case or help me understand why
you need it.
>>
>> Maybe you're only doing this to set a dynamic path for the consumer and producer?
>>
>> Taariq
>>
>> On 29 Dec 2011, at 8:26 PM, David Wynter <david_wynter@yahoo.com> wrote:
>>
>>> Ok, I misunderstood the "Stopping a route during routing an existing message
is a bit tricky. The reason for that is Camel will Graceful Shutdown the route you are stopping.
And if you do that while a message is being routed the Graceful Shutdown will try to wait
until that message has been processed." Which made me think the Processor method described
an alternate way and therefore would stop the route during the message transfer. In fact you
are saying it is an equivalent operation, I think.
>>>
>>> Since I am doing this in code I am not sure which of the many impl of ProcessDefinition
I should be adding to a RouteContext and then adding to the Route.
>>>
>>> Is it like this?
>>>
>>>         RouteDefinition routeD = new RouteDefinition();
>>>                routeD.from(sourcePath);
>>>
>>>          routeD.to(localPath+"?fileExist=Override");
>>>          RouteContext rContext = new DefaultRouteContext(camelContext);
>>>          ProcessorDefinition pDefn = ?
>>>          rContext.createProcessor(pDefn);
>>>          routeD.createProcessor(rContext);
>>>          routeDefn.put(sourcePath+localPath+"?fileExist=Override", routeD);
>>>             camelContext.addRouteDefinition(routeD);
>>>                 camelContext.startRoute(routeD);
>>>
>>> Not sure which ProcessorDefinition type I need.
>>>
>>> Thx.
>>>
>>> David
>>>
>>> ________________________________
>>> From: Taariq Levack <taariql@gmail.com>
>>> To: "users@camel.apache.org" <users@camel.apache.org>
>>> Sent: Thursday, 29 December 2011, 17:50
>>> Subject: Re: NPE for FTP endpoint
>>>
>>> Hmmm, can you talk more about your use case?
>>> Because the FAQ seems a perfect fit, you consume the file and then stop the route,
later it starts again in whatever way you want.
>>>
>>> No idea on the NPE, sorry.
>>>
>>> Taariq
>>>
>>> On 29 Dec 2011, at 7:29 PM, David Wynter <david_wynter@yahoo.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I do not want to stop the route mid flight as that FAQ seems to be suggesting.
I would be happy to use DefaultShutdownStrategy, if it worked. My problem is that it is inconsistent,
which seems bad design, on a slow machine the route is not considered started by the time
I hit the stopRoute, whereas on a fast machine it is considered started. I even used Thread
delays but that did not give consistent results either. Thus I decided to leave the route
running, but that polls the server, a no no with this service provider. There does not seem
a way to just do it once and then be certain the fetch has occurred before stopping the route
to prevent the polling. I'd like to poll until my fetch from the ftp server is complete, and
then stop polling and stop the route, I'll start it again when I am ready. There does not
seem to be a way of adding a listener to the route to ask it, "hey have you actually completed
the route for the file specified?" The only way
>  I
>>>> can think of doing this is adding a custom endpoint as a next step, this
custom endpoint then calls for the route to be stopped.
>>>>
>>>> But these are the least of my worries since on my customers machine I get
the NPE, but in testing here I do not? Any ideas on this NPE?
>>>>
>>>> Thx.
>>>>
>>>> David
>>>>
>>>>
>>>> ________________________________
>>>> From: Taariq Levack <taariql@gmail.com>
>>>> To: "users@camel.apache.org" <users@camel.apache.org>
>>>> Sent: Thursday, 29 December 2011, 17:14
>>>> Subject: Re: NPE for FTP endpoint
>>>>
>>>> Hi
>>>>
>>>> Did you try this FAQ[1] on how to stop a route from a route?
>>>> [1] http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html
>>>>
>>>> Taariq
>>>>
>>>> On 29 Dec 2011, at 6:12 PM, David Wynter <david_wynter@yahoo.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I tested on my machine and deployed to my customer, It went boom as follows,
names etc. changed to protect the innocent.
>>>>>
>>>>> All I want to do with this route is grab a file from a ftp service, once,
don't poll it annoys them, leave the file intact on their server. I will control when this
route is run externally. I tried it with start() follow by stop(),
>>>>>
>>>>>               camelContext.addRouteDefinition(routeD);
>>>>>                   camelContext.startRoute(routeD);
>>>>> but the DefaultShutdownStrategy is flaky, on a slow machine it does not
consider the route started before the stop() is hit, whereas on a fast machine it does. So
I am forced to leave the route running



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/

Mime
View raw message