Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of nishanth.2884@gmail.com
 designates 209.85.192.45 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1423331670272-4184635.post@n3.nabble.com>
References: <1423331670272-4184635.post@n3.nabble.com>
Date: Sun, 8 Feb 2015 12:42:59 -0700
Message-ID: 
 <CAJuNEJCg3QdnWCEVJVMopUyDYVsxdBPihunKzHMaw3QEbT59ww@mail.gmail.com>
Subject: Re: Transaction logs not getting deleted
From: Nishanth S <nishanth.2884@gmail.com>
To: solr-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=047d7bf0cf486f081d050e98dcd5

--047d7bf0cf486f081d050e98dcd5
Content-Type: text/plain; charset=UTF-8

Are you saying that you are not able index at ll for the past  2 days?.Can
you tell me if leaders for all shards are up.There could be large tlog
files when you set auto commit  to a higher value.Check in the logs if
 tlog is getting replayed.I have used only 4.4 and with it the  tlog replay
 was slow.One thing we did was to shut down  solr instance,remove tlog from
the recovering replica and restart(for huge tlogs).It  did a full copy of
the index  from  the leader.It depends on your index size and I/O on how
fast it would be.I am not sure if there is a better solution to this.

This is a very nice blog that I read...
http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/


-Nishanth

On Sat, Feb 7, 2015 at 10:54 AM, vidit.asthana <vidit.asthana7@gmail.com>
wrote:

> Dear Experts,
>
> I have a solrcloud setup - 8 machines, 7 collections(replicationFactor=2,
> numShards=8). Transaction log for one of the replica of a collection is not
> getting deleted and has grown to ~4GB.
>
> Here's the stats for this collection:
>
> *Solr Version:* 4.10.0
> *NumDocs:* 33.5 million
> *Softcommit duration:* 2 minutes
> *Hardcommit duration:* 30 mins
> *Indexing rate:* variable(avg: 2000/sec). Indexing is paused from last 2
> days.
>
> *Size of data directory of this particular buggy replica(shard1_replica1):*
> 20 GB(index dir = 16GB, tlog dir = 4 GB)
> *Size of data directory of second replica of this shard(shard1_replica2):*
> 16 GB(index dir = 16GB, tlog dir = 13 MB)
>
> *Number of files in tlog dir of buggy replica(shard1_replica1) =* 171 (from
> tlog.0000000000000000882 to tlog.0000000000000001172)
>
> Size of most files in tlog directory is 4K. Out of 171 files, 70 are
> greater
> than 1 MB, 10 files are larger than 100MB. Largest tlog file is 935MB.
> Oldest tlog file is 42MB. Latest tlog file in 6MB.
>
> *Number of files in tlog directory of second replica of this
> shard(shard1_replica2) = * 3 (tlog.0000000000000001169,
> tlog.0000000000000001171 and tlog.0000000000000001173)
>
>
> Let me know how can I fix this issue and delete old transaction logs of
> this
> particular replica. I have encountered the same issue previously with 4.8.1
> as well, where tlog kept growing to 20 GB(it was a test collection, so we
> just deleted it at that time).
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Transaction-logs-not-getting-deleted-tp4184635.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

--047d7bf0cf486f081d050e98dcd5--