commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From LeenaB <>
Subject URL verification
Date Thu, 06 Aug 2009 11:18:18 GMT


I'm not sure if it is right forum for my question. But this is the question
for sitemap files required by Google search engine.

My application generates a file having thousands of url to access a various
section of my site. (sitemap urls required by search engine.)
So when I generate this file as it is having thousands of uRS it is
difficult to go and check each and every url if it is up and valid or no.

So I have written one small utility which parse this sitemap url file and
checks if urls are up or no and generates the report.
I use apache commons library for this:

                                     HttpClient client = new HttpClient();
					HttpMethod method = new GetMethod(urlToBeChecked);
					int statusCode = client.executeMethod(method);
					String responseCode=new Integer(statusCode).toString();
						*// add into the list of invalid url*

But this is very slow process; going and hitting each and every url. Is
there any faster method to do the same?

View this message in context:
Sent from the Commons - Issues mailing list archive at

View raw message