lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Furkan KAMACI <furkankam...@gmail.com>
Subject Re: The way Autocommit works in solr - Wierd
Date Mon, 10 Mar 2014 13:57:26 GMT
Hi;

Did you read here:
http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Thanks;
Furkan KAMACI


2014-03-10 15:14 GMT+02:00 RadhaJayalakshmi <rlakshminarayanan@inautix.co.in
>:

> Hi,
>
> Brief Description of my application:
> We have a java program which reads a flat file, and adds document to solr
> using cloudsolrserver.
> And we index for every 1000 documents(bulk indexing).
>
> And the Autocommit setting of my application is:
> <autoCommit>
>         <maxDocs>100000</maxDocs>
>         <openSearcher>false</openSearcher>
> </autoCommit>
>
> So after every 100,000 documents are indexed, engine should perform a
> HardCommit/AutoCommit. But still the OpenSearcher will be false.
> Once the file is fully read, we are issuing a commit() from the
> CloudSolrServer class. So this by default opens a new Searcher.
>
> Also, from the Log, i can see that three times, Autocommit is happenning.
> and Only with the last/final Autocommit, opensearcher is set to true.
>
> So, till now all looks fine and working as expected.
>
> But one strange issue i observed during the course of indexing.
> Now, as per the documentation, the data that is being indexed should first
> get written into tlog. When the Autocommit is performed, the data will be
> flushed to disk.
> So only at three times, there should have been size difference in the
> /index
> folder. All the time only the size of the /tlog folder should have been
> changing
>
> But actually happened is, all the time, i see the size of the /index folder
> getting increased in parallel to the size of the /tlog folder.
> Actually it is increasing to certain limit and coming down. Again
> increasing
> and coming down to a point.
>
> So Now the bigger doubt is have is, during hard commit, is the data getting
> written into both /index or /tlog folder??
>
> I am using solr 4.5.1.
>
> Some one please clear me how the hardcommit works. I am asumming the
> following sequence:
> 1. Reads the data and writes to tlog
> 2. During hardcommit, flushes the data from tlog to index. If openSearcher
> is false, should not open a new searcher
> 3. In the end, once all the datas are indexed, it should open a new
> searcher.
>
> If not please explain me..
>
> Thanks in Advance
> Radha
>
>
>
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/The-way-Autocommit-works-in-solr-Wierd-tp4122558.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message