lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From d_k <mail...@gmail.com>
Subject Re: Viewing the Solr MoinMoin wiki offline
Date Tue, 01 Jan 2013 11:45:21 GMT
I had a partial success with executing a wget as follows:

wget --recursive --page-requisites --no-clobber --html-extension
 --convert-links --restrict-file-names=windows --domains
 wiki.apache.org http://wiki.apache.org/solr/ -w 10 -l 5

configuring a web server to serve that location and then indexing it with:

java -Ddata=web -Ddelay=0 -Drecursive=5 -jar post.jar http://<solr_url>/solr/

I still have some fixes I need to to like decide how to index the
files exactly, maybe a dynamic field that will be indexed and stored.

I can then grep the resulting HTML to point the search box back at the
solr instance.

I didn't test all of it yet, I feel it might work but it feels a little ugly.

On Tue, Jan 1, 2013 at 11:32 AM, Upayavira <uv@odoko.co.uk> wrote:
> I have permission to provide an export. Right now I'm thinking of it
> being a one off dump, without the user dir. If someone wants to research
> how to make moin automate it, I at least promise to listen.
>
> Upayavira
>
> On Tue, Jan 1, 2013, at 08:10 AM, Alexandre Rafalovitch wrote:
>> That's why I think this could be a nice joint project with Apacha Infra.
>> They provide Moin export, we build a way to index it with Solr for local
>> usage. Start with our own - Solr - project , then sell it to others once
>> it
>> has been dog-fooded enough. Instant increased Solr exposure to all Apache
>> project users.....
>>
>> Just a thought.
>>
>> Regards,
>>    Alex.
>>
>> Personal blog: http://blog.outerthoughts.com/
>> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
>> - Time is the quality of nature that keeps events from happening all at
>> once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)
>>
>>
>> On Tue, Jan 1, 2013 at 7:03 PM, Lance Norskog <goksron@gmail.com> wrote:
>>
>> > 3 problems:
>> > a- he wanted to read it locally.
>> > b- crawling the open web is imperfect.
>> > c- /browse needs to get at the files with the same URL as the uploader.
>> >
>> > a and b- Try downloading the whole thing with 'wget'. It has a 'make links
>> > point to the downloaded files' option. Wget is great.
>> >
>> > I have done this by parking my files behind a web server. You can use
>> > Tomcat. (I recommend the XAMPP distro: http://www.apachefriends.org/**
>> > en/xampp.html <http://www.apachefriends.org/en/xampp.html>). Then, use
>> > Erik's command to crawl that server. Use /browse to read it.
>> >
>> > Looking at this again, it should be possible to add a file system service
>> > to the Solr start.jar etc/jetty.xml file. I think I did this once. It would
>> > be a handy patch. In fact, this whole thing would make a great blog post.
>> >
>> >
>> > On 12/30/2012 05:05 AM, Erik Hatcher wrote:
>> >
>> >> Here's a geeky way to do it yourself:
>> >>
>> >> Fire up Solr 4.x, run this from example/exampledocs:
>> >>
>> >>     java -Ddata=web -Ddelay=2 -Drecursive=1 -jar post.jar
>> >> http://wiki.apache.org/solr/
>> >>
>> >> (although I do end up getting a bunch of 503's, so maybe this isn't very
>> >> reliable yet?)
>> >>
>> >> Tada: http://localhost:8983/solr/**collection1/browse<http://localhost:8983/solr/collection1/browse>
>> >>
>> >> :)
>> >>
>> >>         Erik
>> >>
>> >>
>> >> On Dec 29, 2012, at 16:54 , d_k wrote:
>> >>
>> >>  Hello,
>> >>>
>> >>> I'm setting up Solr inside an intranet without an internet access and
>> >>> I was wondering if there is a way to obtain the data dump of the Solr
>> >>> Wiki (http://wiki.apache.org/solr/) for offline viewing and searching.
>> >>>
>> >>> I understand MoinMoin has an export feature one can use
>> >>> (http://moinmo.in/MoinDump and
>> >>> http://moinmo.in/**HelpOnMoinCommand/ExportDump<http://moinmo.in/HelpOnMoinCommand/ExportDump>)
>> >>> but i'm afraid it needs
>> >>> to be executed from within the MoinMoin server.
>> >>>
>> >>> Is there a way to obtain the result of that command?
>> >>> Is there another way to view the solr wiki offline?
>> >>>
>> >>
>> >

Mime
View raw message