commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From LeenaB <lee...@synechron.com>
Subject URL verification
Date Thu, 06 Aug 2009 11:18:18 GMT

Hi,

I'm not sure if it is right forum for my question. But this is the question
for sitemap files required by Google search engine.

My application generates a file having thousands of url to access a various
section of my site. (sitemap urls required by search engine.)
So when I generate this file as it is having thousands of uRS it is
difficult to go and check each and every url if it is up and valid or no.

So I have written one small utility which parse this sitemap url file and
checks if urls are up or no and generates the report.
I use apache commons library for this:

{code}
                                     HttpClient client = new HttpClient();
					HttpMethod method = new GetMethod(urlToBeChecked);
					int statusCode = client.executeMethod(method);
					String responseCode=new Integer(statusCode).toString();
					if(!responseCode.startsWith("2")){
						*// add into the list of invalid url*
					}
					method.releaseConnection();
{code}

But this is very slow process; going and hitting each and every url. Is
there any faster method to do the same?

Regard,
Leena
-- 
View this message in context: http://www.nabble.com/URL-verification-tp24844563p24844563.html
Sent from the Commons - Issues mailing list archive at Nabble.com.


Mime
View raw message