lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopal Patwa <gopalpa...@gmail.com>
Subject Re: Zookeeper exceptions - SEVERE
Date Wed, 19 Mar 2014 04:42:01 GMT
Shalin, "correlated with how frequently you call commit" is it soft commit
or hard commit? , I guess it should be later one.

just curious what data it update to zookeeper during commit


On Tue, Mar 18, 2014 at 9:12 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:

> SolrCloud will update Zookeeper on state changes (node goes to
> recovery, comes back up etc) or for leader election and during
> collection API commands. It doesn't correlate directly with indexing
> but is correlated with how frequently you call commit.
>
> On Wed, Mar 19, 2014 at 5:46 AM, Shawn Heisey <solr@elyograg.org> wrote:
> > On 3/18/2014 5:46 PM, Chris W wrote:
> >>
> >> I am running a 3 node zookeeper 3.4.5  Quorum. I am running into issues
> >> with Zookeeper transaction logs
> >>
> >>   [myid:2] - ERROR [main:QuorumPeer@453] - Unable to load database on
> disk
> >> java.io.IOException: Unreasonable length = 1048587
> >> at
> >>
> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:100)
> >> at
> >> org.apache.zookeeper.server.persistence.Util.readTxnBytes(Util.java:233)
> >> at
> >>
> >>
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:602)
> >> at
> >>
> >>
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:157)
> >> at
> >> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> >>
> >> To unblock temporarily, i deleted the most recent txn log. How do i tell
> >> zookeeper to not grow the transaction log beyond x MegaBytes?
> >>
> >> How often does the transaction log get updated? Does zk transactions log
> >> grow everytime we index data into a new collection?
> >
> >
> > Zookeeper is a separate project at Apache.  ZK file management is
> discussed
> > in the ZK documentation.
> >
> >
> http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
> >
> > There is a bug filed on Zookeeper for the issue you are seeing, with a
> > fairly simple workaround.  It is fixed in the 3.4.6 version, which was
> > released last week.  I will see whether we can get ZK upgraded to 3.4.6
> in
> > the Solr 4.8.0 release.  I don't think we want to risk doing that
> upgrade in
> > 4.7.1, but I could be wrong.
> >
> > https://issues.apache.org/jira/browse/ZOOKEEPER-1513
> > http://zookeeper.apache.org/releases.html
> >
> > I am actually not sure how often SolrCloud updates Zookeeper.  It happens
> > whenever the collections API is called for sure, and it may happen
> anytime
> > you index data as well.
> >
> > Thanks,
> > Shawn
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message