lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shalin Shekhar Mangar <shalinman...@gmail.com>
Subject Re: Zookeeper exceptions - SEVERE
Date Wed, 19 Mar 2014 04:45:04 GMT
Sorry guys I spoke too fast. I looked at the code again. No it doesn't
correlate with commits at all. I was mistaken.

On Wed, Mar 19, 2014 at 10:06 AM, Chris W <chris1980.sql@gmail.com> wrote:
> Thanks, Shawn and Shalin
>
> How does the frequency of commit affect zookeeper?
>
>
> Thanks
>
>
> On Tue, Mar 18, 2014 at 9:12 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
>> SolrCloud will update Zookeeper on state changes (node goes to
>> recovery, comes back up etc) or for leader election and during
>> collection API commands. It doesn't correlate directly with indexing
>> but is correlated with how frequently you call commit.
>>
>> On Wed, Mar 19, 2014 at 5:46 AM, Shawn Heisey <solr@elyograg.org> wrote:
>> > On 3/18/2014 5:46 PM, Chris W wrote:
>> >>
>> >> I am running a 3 node zookeeper 3.4.5  Quorum. I am running into issues
>> >> with Zookeeper transaction logs
>> >>
>> >>   [myid:2] - ERROR [main:QuorumPeer@453] - Unable to load database on
>> disk
>> >> java.io.IOException: Unreasonable length = 1048587
>> >> at
>> >>
>> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:100)
>> >> at
>> >> org.apache.zookeeper.server.persistence.Util.readTxnBytes(Util.java:233)
>> >> at
>> >>
>> >>
>> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:602)
>> >> at
>> >>
>> >>
>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:157)
>> >> at
>> >> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>> >>
>> >> To unblock temporarily, i deleted the most recent txn log. How do i tell
>> >> zookeeper to not grow the transaction log beyond x MegaBytes?
>> >>
>> >> How often does the transaction log get updated? Does zk transactions log
>> >> grow everytime we index data into a new collection?
>> >
>> >
>> > Zookeeper is a separate project at Apache.  ZK file management is
>> discussed
>> > in the ZK documentation.
>> >
>> >
>> http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
>> >
>> > There is a bug filed on Zookeeper for the issue you are seeing, with a
>> > fairly simple workaround.  It is fixed in the 3.4.6 version, which was
>> > released last week.  I will see whether we can get ZK upgraded to 3.4.6
>> in
>> > the Solr 4.8.0 release.  I don't think we want to risk doing that
>> upgrade in
>> > 4.7.1, but I could be wrong.
>> >
>> > https://issues.apache.org/jira/browse/ZOOKEEPER-1513
>> > http://zookeeper.apache.org/releases.html
>> >
>> > I am actually not sure how often SolrCloud updates Zookeeper.  It happens
>> > whenever the collections API is called for sure, and it may happen
>> anytime
>> > you index data as well.
>> >
>> > Thanks,
>> > Shawn
>> >
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
>
>
> --
> Best
> --
> C



-- 
Regards,
Shalin Shekhar Mangar.

Mime
View raw message