zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharat Singh <bharatsingh.mnni...@gmail.com>
Subject Re: Why 1K filesize limit in ZKfuse
Date Sat, 13 Jun 2015 18:40:50 GMT
Thanks Camille.

Yes I agree, It will take more memory and replication time might increase.
I am more interested if there are any possible issues if we operate with
the max allowed file size or as we increase the no of files. Do we have any
benchmarking/performance stats for this ?

How do i decide a safe limit.


On Sat, Jun 13, 2015 at 11:02 AM, Camille Fournier <camille@apache.org>
wrote:

> I'm not sure that there is a reason for it honestly. Possibly no one ever
> used it for anything larger than 1k. I don't know the package well myself
> but assuming it's storing this data inside of zk nodes having large files
> in there will fill up the zk memory fast and may be risky, but other than
> that I can't think of a reason.
>
> Best
> C
> On Jun 13, 2015 1:52 PM, "Bharat Singh" <bharatsingh.mnnit06@gmail.com>
> wrote:
>
> > Ping ..
> >
> > On Wed, Jun 10, 2015 at 5:01 PM, Bharat Singh <
> > bharatsingh.mnnit06@gmail.com
> > > wrote:
> >
> > > Hi,
> > >
> > > I am trying to use zookeeper with zkfuse interface. I have some doubts
> > > regarding max file size supported.
> > >
> > > Zookeeper max file size: (1MB)
> > >
> > >
> >
> http://zookeeper.apache.org/doc/r3.1.2/api/org/apache/zookeeper/ZooKeeper.htm
> > >
> > > The maximum allowable size of the data array is 1 MB
> > >
> > > ZKfuse max file size: (1KB)
> > >
> > >
> >
> http://code.metager.de/source/xref/apache/zookeeper/src/contrib/zkfuse/src/zkfuse.cc
> > > *static* *const* *unsigned* maxDataFileSize
> > > <http://code.metager.de/source/s?refs=maxDataFileSize&project=apache>
> =
> > > MAX_DATA_SIZE
> > > <
> >
> http://code.metager.de/source/xref/apache/zookeeper/src/contrib/zkfuse/src/zkfuse.cc#MAX_DATA_SIZE
> > >
> > > ;
> > >
> > >
> > > Why ZKfuse reduces the size further since we already have a cap at
> > > zookeeper.
> > > Is there any explanation behind this check in ZKfuse.
> > >
> > > Please point me to appropriate forum if this is not the one.
> > >
> > > Suggestions are appreciated.
> > >
> > >
> > > Thanks,
> > > Bharat
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message