camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From garyk70 <gkirr...@gmail.com>
Subject How to determine when one's streamed to the end of a file?
Date Tue, 29 Jun 2010 05:27:31 GMT

Good evening,

I'm attempting to stream a large CSV file, split on each row, aggregrate the
rows in batches, and insert the batched rows into a database table.  My only
issue is that I can't figure out how to determine when I've reached the end
of the file (and therefore the end of the aggregations and inserts).  Is
there a ready means of detecting the end of the file (and if not, perhaps a
more effective route design than mine)?

Here's my route:

from("file:///Documents/tmp?fileName=TEST.dat")
.process(new Processor() {
	public void process(Exchange exchange) throws Exception {
		dao.truncateTable();
		dao.disableUniqueKeyIndex();
	}
}).split(body().tokenize()).streaming().aggregate(header("CamelFileName"),
new AggregationStrategy() {

	@SuppressWarnings("unchecked")
	public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {

		if (oldExchange == null) {
			oldExchange = newExchange.copy();
			oldExchange.getIn().setBody(new ArrayList());
		}

		List<String> rows = oldExchange.getIn().getBody(List.class);
		String row = newExchange.getIn().getBody(String.class);
		rows.add(row);

		return oldExchange;
	}
}).completionSize(1000).completionTimeout(30000).process(new Processor() {

	public void process(Exchange exchange) throws Exception {

		List<String> rows = exchange.getIn().getBody(List.class);
		dao.loadDatFileRows(rows);
	}
// read pretend "Status" property
}).end().filter(property("STATUS").isEqualTo("FINISHED")).process(new
Processor() {

	public void process(Exchange exchange) throws Exception {
		dao.rebuildUniqueKeyIndex();
		dao.updateStatistics();
	}
});

Thanks in advance for any guidance.
-- 
View this message in context: http://camel.465427.n5.nabble.com/How-to-determine-when-one-s-streamed-to-the-end-of-a-file-tp511784p511784.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mime
View raw message