manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject WebSite Crawling with Customheader Info
Date Wed, 05 Dec 2018 09:54:53 GMT
Can you please suggest is there a way to pass custom header info with Seed URL 
so that the target application can determine this request is coming for crawling 
i.e if my site is so when the request hits from ManifoldCF for crawling
can i pass some header which I can parse in my  site to determine
the request is for crawling - and I can customize my code for some purposes
like - Bypass Authentication - The reason I am asking is for some reason
I could not map my login sequence defined in ManifoldCF - so looking for alternatives
(Since it's my Internet Site - Its ok for me to ByPass Authentication based on some
custom header)

Thanks In Advance for help

View raw message