camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Wolf <cwolf.a...@gmail.com>
Subject Re: How to schedule FTP with quartz?
Date Wed, 27 Mar 2013 02:00:42 GMT
Pontus,

Ok, I got rid of my RoutePolicy and put back the
CronScheduledRoutePolicy and provided both start and resume
cron schedules as you suggested, but CronScheduledRoutePolicy has the
same issue as my RoutePolicy - suspending
the FTPConsumer does NOT stop it from continuing to poll!

56:30,001 CronScheduledRoutePolicy       DEBUG Suspended consumer
FtpConsumer[ftp://localhost/download?filter=%23cpmdFileFilter&noop=true&password=******&username=adpt5]
56:35,100 FtpConsumer                    WARN  Cannot connect/login
to: ftp://adpt5@localhost:21. Will skip this poll.
56:35,100 FtpConsumer                    DEBUG Skipping poll as pre
poll check returned false


   -Chris

On Tue, Mar 26, 2013 at 4:12 PM, Pontus Ullgren <ullgren@gmail.com> wrote:
> Chris,
>
> Is there no way for you to calculate a cron expression for when the
> suspend should occure ?
>
> Say that you want the route to start every 10 minutes (starting at 0)
> and run for 5 minutes then suspend.
> This would mean that you should be able to define the start/resume
> cron as "0 0,10,20,30,40,50 * * * * ?".
> And the suspend cron expression as "0 5,15,25,35,45,55 * * * * ?".
>
> The CronScheduledRoutePolicy will trigger a start/resum at 00:00:00
> and then a suspend at 00:05:00, then a new start/resume at 00:10:00
> and a new suspends at 00:15:00 and so on.
>
> // Pontus
>
> On Tue, Mar 26, 2013 at 7:39 PM, Chris Wolf <cwolf.algo@gmail.com> wrote:
>> Pontus,
>>
>> I actually gave up on CronScheduledRoutePolicy because I don't want to
>> have to calculate an absolute suspend time based on the start/resume
>> time.  What I need is a cron-based policy that specifies the initial
>> start time, which there-after becomes the resume time - this is a cron
>> expression - then I just want a relative run duration, after which,
>> the route is suspended until the next cron-specified resume time.
>>
>> So, after a few days, I finally have that and it works in the unit
>> test - even the initial one-off route start will transparently switch
>> to a resume schedule.  The only problem is that I really need this
>> policy to control a route with an FTP consumer.  What I'm seeing is
>> that
>> even though the route is suspended, the FTP consumer continues to poll
>> - this partially answers my question about the coding of
>> ScheduleRoutePolicy - which only suspends the consumer - not the
>> route, itself.   While my policy suspends the route.
>>
>> I suspect that any route which has a component using
>> PollingConsumerPollStrategy will not behave as I though - which is -
>> you only need to suspend the route and all it's components will be
>> suspended.  My suspicion is that components whose Consumers are under
>> the influence of PollingConsumerPollStrategy may not suspend by only
>> suspended the route.
>>
>>    -Chris
>>
>>
>> On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ullgren@gmail.com> wrote:
>>> This is probably because your route is autoStart=false. So you the first
>>> time you need to start it. In my example you see I set both the start and
>>> resume schedule to the same cron expression. So the policy will trigger
>>> both a start and a resume action.
>>>
>>> And you will get a WARN log since the first time it can not resume (but it
>>> will start) and after that it can not start but it will resume.
>>>
>>> Perhaps if you share some code it would be easier to help you.
>>> //Pontus
>>> On 22 Mar 2013 22:06, "Chris Wolf" <cwolf.algo@gmail.com> wrote:
>>>
>>>> I found the issue with my custom CronScheduledRoutePolicy - initially
>>>> the startTime/resumeTime are only scheduled in
>>>> onInit() - so to re-resume (re-start), you need to call
>>>> scheduleRoute(Action.RESUME, route); in onStart()
>>>>
>>>> ...but now I'm getting:
>>>>
>>>> quartz.ScheduledRoutePolicy WARN  Route is not in a started state and
>>>> cannot be resumed. The current route state is Suspended
>>>>
>>>> What is the deal?  I thought resumeRoute was the inverse of
>>>> suspenRoute, but this log message seems to indicate that
>>>> calling CamelContext.suspendRoute(routeId) will put the route into a
>>>> state that cannot be resumed.
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Chris
>>>>
>>>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cwolf.algo@gmail.com>
wrote:
>>>> > Pontus,
>>>> >
>>>> > Thanks for that.  Since I already has started implementing a class
>>>> > derived from CronScheduledRoutePolicy, I just finished it.
>>>> > It works by starting a Timer thread in onStart/onResume at the end of
>>>> > the time period, the route is suspended, but then upon
>>>> > the next schedule cron start time, I don't see it being resumed - I
>>>> > wonder if the RoutePolicy itself is being suspend too?
>>>> >
>>>> > Well, I try it your way also.
>>>> >
>>>> >
>>>> > Thanks,
>>>> >
>>>> >
>>>> > Chris
>>>> >
>>>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ullgren@gmail.com>
>>>> wrote:
>>>> >> Hello,
>>>> >>
>>>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cwolf.algo@gmail.com>
>>>> wrote:
>>>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ullgren@gmail.com>
>>>> wrote:
>>>> >>>> Hello Chris,
>>>> >>>>
>>>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cwolf.algo@gmail.com>
>>>> wrote:
>>>> >>>>> Claus,
>>>> >>>>>
>>>> >>>>> I have a few further questions about CronScheduledRoutePolicy.
 I
>>>> >>>>> noticed that it has setters such as setRouteStartTime,
>>>> >>>>> setRouteStopTime, each which takes a cron expression
string.  What
>>>> I'm
>>>> >>>>> looking for is to be able to use a cron expression for
the start, but
>>>> >>>>> a relative time length for stop.   Otherwise, I need
to write code to
>>>> >>>>> parse the start time expression, then calculate a stop
time cron
>>>> >>>>> expression.  Any ideas?
>>>> >>>>>
>>>> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle"
on
>>>> >>>> the endpoint and then suspend the route when you receive
a empty
>>>> >>>> message. Which means that there is no more files to poll
at the
>>>> >>>> moment.
>>>> >>>> You can use the content based route EIP for this.
>>>> >>>
>>>> >>> That is interesting to know, thanks.  In my case, the files
at the
>>>> >>> remote end are themselves deposited at an irregular rate, but
within a
>>>> >>> defined time window, so during that time window, there will
be
>>>> >>> intermittent idleness...
>>>> >>>>
>>>> >>>> Another solution would be to write your own RoutePolicy
to take care
>>>> >>>> of your needs.
>>>> >>>
>>>> >>> Yes, this sounds like the best approach...
>>>> >>>
>>>> >>>
>>>> >>>>
>>>> >>>> I just started to wonder if it might be possible to combine
the
>>>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>>> >>>> I have _not_ tested this so I'm not sure if it works. It
might be that
>>>> >>>> there is a collision in the way they work with Quartz.
>>>> >>>>
>>>> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>>> >>>>> setRouteSuspendTime such that for my FTP poll window,
I could either
>>>> >>>>> do start/stop or resume/suspend - which is recommended?
>>>> >>>>>
>>>> >>>> I would highly recommend resume/suspend.
>>>> >>>> I've had some thread leak problem with the file component
when it was
>>>> >>>> repetitively started/stopped.
>>>> >>>
>>>> >>> Ok, but I guess the first policy callback with be onStart since
the
>>>> >>> route will be
>>>> >>> configured with noAutoStartup(), so upon that first onStart,
I'll
>>>> >>> suspend then toggle
>>>> >>> between onSuspend/onResume...
>>>> >>>
>>>> >> Yes this is what I do. We have a route that should start 06:30 each
>>>> >> day and then poll all files that are in the folder at that time.
>>>> >> After that it should suspend.
>>>> >>
>>>> >> Here is some pseudo code.
>>>> >> SuspendRouteProcessor is a processor that suspends the route based
on
>>>> route id.
>>>> >> ---
>>>> >> String cronStr = "* 30 6 * * * ?";
>>>> >> String input = "ftp://user@remotehost
>>>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
>>>> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>>>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
>>>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>>>> >>
>>>> >> from(input)
>>>> >>     .routeId("input1")
>>>> >>         .routePolicy(versionPolicy, scheduledRoutePolicy)
>>>> >>         .noAutoStartup()
>>>> >>          .choice()
>>>> >>             .when(body().isNotNull())
>>>> >>                    .to("direct:processFiles")
>>>> >>                 .end()
>>>> >>                 .log(LoggingLevel.DEBUG, "All files processed, suspend
>>>> route")
>>>> >>                 .process(new SuspendRouteProcessor("input1"))
>>>> >>         ;
>>>> >> --
>>>> >>
>>>> >> The only downside with this is that after the initial start we get
a
>>>> >> WARN log message that the route can not be started since it is in
>>>> >> suspend state.
>>>> >> But as long as you can live with the WARN log it works.
>>>> >>
>>>> >>>>
>>>> >>>> // Pontus
>>>> >>>>
>>>> >>>>> Thanks,
>>>> >>>>>
>>>> >>>>> Chris
>>>> >>>>>
>>>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <claus.ibsen@gmail.com>
>>>> wrote:
>>>> >>>>>> Hi
>>>> >>>>>>
>>>> >>>>>> See about route policy
>>>> >>>>>> http://camel.apache.org/routepolicy
>>>> >>>>>>
>>>> >>>>>> And the scheduled route policy
>>>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>> >>>>>>
>>>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cwolf.algo@gmail.com>
>>>> wrote:
>>>> >>>>>>> I have a requirement to download files via FTP
during a certain
>>>> time
>>>> >>>>>>> window and according to a schedule. e.g. Only
on trading days
>>>> between
>>>> >>>>>>> 6:30AM and 7:00AM.  The FTP component, alone,
seems to just do
>>>> >>>>>>> indefinite polling according to delay/initialDelay.
>>>> >>>>>>>
>>>> >>>>>>> From the "Camel In Action" book, chapter 7,
I see some examples of
>>>> >>>>>>> sending a text message with the Timer and Quartz
components, but I
>>>> >>>>>>> can't quite see how to put that together to
implement "kicking off
>>>> >>>>>>> routes at specified intervals", mentioned in
the best practices
>>>> list
>>>> >>>>>>> at the end of that chapter.  How would I use
quartz to stop/start
>>>> the
>>>> >>>>>>> FTP component, or the route that it's in?
>>>> >>>>>>>
>>>> >>>>>>> Thanks,
>>>> >>>>>>>
>>>> >>>>>>> Chris
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Claus Ibsen
>>>> >>>>>> -----------------
>>>> >>>>>> Red Hat, Inc.
>>>> >>>>>> FuseSource is now part of Red Hat
>>>> >>>>>> Email: cibsen@redhat.com
>>>> >>>>>> Web: http://fusesource.com
>>>> >>>>>> Twitter: davsclaus
>>>> >>>>>> Blog: http://davsclaus.com
>>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>

Mime
View raw message