camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jpcook <jonathan.c...@erars.plus.com>
Subject Re: Atom Component
Date Fri, 23 Oct 2009 14:10:04 GMT

Hello,

I ended up solving this problem with the follow configure method:

private void configureNewsTrackerByTeamHtml() throws Exception {
		
		for(String url : urls){			
			from("atom://"+url+"?splitEntries=false&consumer.delay="+consumerDelay)
				.to("seda:footballNewsTrackerByTeamHtml");
		}		
		
		from("seda:footballNewsTrackerByTeamHtml").convertBodyTo(String.class)			
			.process(new SearchFeedProcessor()).convertBodyTo(String.class)		
			.process(XsltBuilder.xslt(new
File("xslt//football//newstracker//html//football_newstracker.html.xslt")))		
			.setHeader(FileComponent.HEADER_FILE_NAME, 
					constant("/feeds/football/teams/")
					.append(header(FeedConsumerConstants.SEARCHFEED_TEAM_ID))
					.append("/news_tracker.inc"))                
		
.to("file://#config(output.content)?append=false&noop=true&fileEncoding=ISO-8859-1");
	}

Not sure if this has many disadvantages over the solution you proposed using
a pojo and consumer template but it works nicely?

Thanks


Claus Ibsen-2 wrote:
> 
> On Mon, Oct 5, 2009 at 5:01 PM, jpcook <jonathan.cook@erars.plus.com>
> wrote:
>>
>> Hi,
>>
>> I was also wondering why you can't specify that you want all the entries
>> sorted? It appears that sortEntries only works when splitEntries is true?
> 
> Create a ticket and we love contributions. So please try to fix it
> yourself and submit a patch to us.
> 
>>
>> Thanks
>>
>>
>> jpcook wrote:
>>>
>>> Thanks. Useful to know.
>>> We aren't using maven unfortunately.
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> On Wed, Sep 30, 2009 at 3:57 PM, jpcook <jonathan.cook@erars.plus.com>
>>>> wrote:
>>>>>
>>>>> Upgraded to the latest latest version 0.4.0 and it works. Might be
>>>>> useful to
>>>>> list the dependancies on the wiki? Camel just eats the exception as
>>>>> well.
>>>>
>>>> Most people use maven and have the dependencies for free.
>>>>
>>>> If you are not then you have to check this out yourself.
>>>> For example see the pom.xml file for the the component which lists the
>>>> dependencies it uses.
>>>>
>>>> Or see the maven reposts (which can be old as they are not updated very
>>>> often)
>>>> http://camel.apache.org/maven/camel-atom/dependency-convergence.html
>>>> http://camel.apache.org/maven/index.html
>>>>
>>>>
>>>>>
>>>>>
>>>>> jpcook wrote:
>>>>>>
>>>>>> Got further.
>>>>>>
>>>>>> What version of abdera does this work with. I have downloaded the
>>>>>> latest
>>>>>> which is 0.3.0 and it doesn't seem to contain the getInstance method
>>>>>> used
>>>>>> here.
>>>>>>
>>>>>> Abdera.getInstance().getParser();
>>>>>>
>>>>>> if I use Abdera.getNewParser() it works ok.
>>>>>>
>>>>>> Any ideas please?
>>>>>>
>>>>>>
>>>>>> jpcook wrote:
>>>>>>>
>>>>>>> Strange just using the http endpoint works fine.
>>>>>>>
>>>>>>>
>>>>>>> jpcook wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> So just trying a very simple example like:
>>>>>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>>>>>
>>>>>>>> But it returns:
>>>>>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom -
>>>>>>>> Exchange[BodyType:null,
>>>>>>>> Body:null]
>>>>>>>>
>>>>>>>> No errors in the console so not sure what I'm doing wrong?
>>>>>>>>
>>>>>>>>
>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>
>>>>>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook
>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>> One last probably silly question. What you initialise
>>>>>>>>>> consumerTemplate
>>>>>>>>>> as or
>>>>>>>>>> do you extend from a particular class for this to
work?
>>>>>>>>>>
>>>>>>>>>> We're not using Spring so can't do any injection
as per another
>>>>>>>>>> thread
>>>>>>>>>> I
>>>>>>>>>> saw.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> You can create a consumer template from the CamelContext
>>>>>>>>> And then you can reuse it on subsequent invocations.
>>>>>>>>>
>>>>>>>>> If you use a bean/pojo you can just have a CamelContext
parameter
>>>>>>>>> in
>>>>>>>>> the method signature and Camel will inject the context
for you.
>>>>>>>>>
>>>>>>>>> If you use a processor then you can get hold of the CamelContext
>>>>>>>>> from
>>>>>>>>> the Exchange using getContext()
>>>>>>>>>
>>>>>>>>> public void doSomething(CamelContext context) {
>>>>>>>>> if (consumerTemplate == null) {
>>>>>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> And then does the url actually contain the atom part?
Looking at
>>>>>>>>>> the
>>>>>>>>>> wiki
>>>>>>>>>> page it would indicate it should do eg) Exchange
exchange =
>>>>>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yeah you need to use the complete URL
>>>>>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> thanks
>>>>>>>>>>>>
>>>>>>>>>>>> The atom endpoint already has consumer.delay
which I think I
>>>>>>>>>>>> would
>>>>>>>>>>>> use.
>>>>>>>>>>>
>>>>>>>>>>> Yeah but the atom endpoint will then be configured
with a static
>>>>>>>>>>> endpoint
>>>>>>>>>>> URI
>>>>>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>>>>>
>>>>>>>>>>> Where as if you use a processor/bean/ with a
consumer template
>>>>>>>>>>> you
>>>>>>>>>>> can do
>>>>>>>>>>>
>>>>>>>>>>> from("timer://foo?delay=5000").beanRef("myBean",
"doSomething)
>>>>>>>>>>>
>>>>>>>>>>> And then in your POJO you can poll the atom endpoint
using a
>>>>>>>>>>> dynamic
>>>>>>>>>>> URI
>>>>>>>>>>>
>>>>>>>>>>> private List<String> uris;
>>>>>>>>>>>
>>>>>>>>>>> public void doSomething() {
>>>>>>>>>>>    // loop the list of dynamic uris and get
the content from it
>>>>>>>>>>>   // and then consume from the endpoint using
consumer template
>>>>>>>>>>>   Exchange out = consumerTemplate.receive(uri,
1000);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> And see that link Charles mentioned.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>>>>>
>>>>>>>>>>>> Could I use the web-console to configure
the dynamic uris or
>>>>>>>>>>>> just
>>>>>>>>>>>> a
>>>>>>>>>>>> normal
>>>>>>>>>>>> xml configuration file?
>>>>>>>>>>>>
>>>>>>>>>>>> About the last point, that is how I have
done it in the past
>>>>>>>>>>>> but
>>>>>>>>>>>> just
>>>>>>>>>>>> thought I'd check if there was something
built into camel now.
>>>>>>>>>>>> :)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can use a scheduler / timer to trigger
a route at a
>>>>>>>>>>>>> certain
>>>>>>>>>>>>> interval (quartz or timer)
>>>>>>>>>>>>>
>>>>>>>>>>>>> And then use a processor / bean with
a consumerTemplate to
>>>>>>>>>>>>> consume
>>>>>>>>>>>>> from the atom feeds.
>>>>>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> And if you want that to route in parallel
you can use the JDK
>>>>>>>>>>>>> concurrency API for that as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sometimes the easiest stuff is to do
that using regular java
>>>>>>>>>>>>> in
>>>>>>>>>>>>> a
>>>>>>>>>>>>> POJO.
>>>>>>>>>>>>> Submit tasks to the JDK executor services
and then afterwards
>>>>>>>>>>>>> route
>>>>>>>>>>>>> the result to a file endpoint to store
the file.
>>>>>>>>>>>>> Or a "direct" endpoint so you can do
additional routing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a requirement to pull 24 atom
feeds, process them in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> same
>>>>>>>>>>>>>> way
>>>>>>>>>>>>>> via
>>>>>>>>>>>>>> xslt and then write the results to
a file location which is
>>>>>>>>>>>>>> slightly
>>>>>>>>>>>>>> different for each feed. This is
fairly straight forward.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was looking at the atom component
as it looks almost
>>>>>>>>>>>>>> perfect.
>>>>>>>>>>>>>> But I
>>>>>>>>>>>>>> wondered if there was a clever way
I could maybe specify a
>>>>>>>>>>>>>> list
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> the component and then it could process
them concurrently as
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>> want
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> have to do this synchronously? A
bit like when you use the
>>>>>>>>>>>>>> splitter you
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I guess another alternative would
be to have a route for each
>>>>>>>>>>>>>> feed
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> pull but this seemed a bit overkill
as they would essentially
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>> all
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> same. Also I wanted to make the atom
urls and the location
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> result
>>>>>>>>>>>>>> gets written to configurable but
we are not using the spring
>>>>>>>>>>>>>> dsl
>>>>>>>>>>>>>> xml
>>>>>>>>>>>>>> configuration. As an alternative
I could make these
>>>>>>>>>>>>>> parameters
>>>>>>>>>>>>>> configurable
>>>>>>>>>>>>>> via my own configuration but I also
wondered if I could
>>>>>>>>>>>>>> perhaps
>>>>>>>>>>>>>> control
>>>>>>>>>>>>>> these parameters via JMX or even
better via the web console?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>>>>>> Sent from the Camel - Users mailing
list archive at
>>>>>>>>>>>>>> Nabble.com.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>>>
>>>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>>>>>> Sent from the Camel - Users mailing list
archive at Nabble.com.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>
>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive
at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Atom-Component-tp25609495p25681591.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Atom-Component-tp25609495p25752445.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> Claus Ibsen
> Apache Camel Committer
> 
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
> 
> 

-- 
View this message in context: http://www.nabble.com/Atom-Component-tp25609495p26027107.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message