incubator-blur-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: Auto-save of block-cache....
Date Mon, 22 Jun 2015 12:39:59 GMT
On Wed, May 20, 2015 at 4:52 AM, Ravikumar Govindarajan <
ravikumar.govindarajan@gmail.com> wrote:

> >
> > Do you have any feel
> > for performance of the cache warmer?  Meaning if a shard server fails do
> > you have any metrics on performance differences before the cache warmer
> vs
> > without?
>
>
> I just quickly drafted a prototype to get your views on this. I am yet to
> run unit-tests and surely don't have any stats as of now...
>
> My gut feel is that cache-warmups will take around 1-2 hours for
> non-trivial cache-sizes {60-70GB} with 2-4 MB/sec read throttle speeds.
> Some apps may find no value addition with this.
>
> May be we can make this optional via blur-config? Users observing no
> improvements or even detrimental to existing code can switch this off…
>

Yeah we should do that.  Also we haven't forgotten about this feature, just
have been busy working off critical issues for 0.2.4 release.

Aaron


>
> --
> Ravi
>
> On Tue, May 19, 2015 at 6:38 PM, Aaron McCurry <amccurry@gmail.com> wrote:
>
> > On Mon, May 18, 2015 at 7:56 AM, Ravikumar Govindarajan <
> > ravikumar.govindarajan@gmail.com> wrote:
> >
> > > Aaron,
> > >
> > > I've re-worked the first-cut patch to make it more simpler. If you
> could
> > go
> > > through it when you find time, it'll be great…
> > >
> >
> > I just read through the code.  I like it!  Thanks!  Do you have any feel
> > for performance of the cache warmer?  Meaning if a shard server fails do
> > you have any metrics on performance differences before the cache warmer
> vs
> > without?  Thanks.
> >
> > Aaron
> >
> >
> > >
> > > --
> > > Ravi
> > >
> > > On Thu, May 7, 2015 at 3:43 PM, Aaron McCurry <amccurry@gmail.com>
> > wrote:
> > >
> > > > Thanks!  Will do!
> > > >
> > > > On Wednesday, May 6, 2015, Ravikumar Govindarajan <
> > > > ravikumar.govindarajan@gmail.com> wrote:
> > > >
> > > > > Done… https://issues.apache.org/jira/browse/BLUR-431
> > > > >
> > > > > Please do review when you find time…
> > > > >
> > > > > --
> > > > > Ravi
> > > > >
> > > > > On Wed, May 6, 2015 at 7:30 AM, Aaron McCurry <amccurry@gmail.com
> > > > > <javascript:;>> wrote:
> > > > >
> > > > > > Hey Ravi,
> > > > > >
> > > > > > Attachments are removed from the mail list.  Perhaps you can
> create
> > > an
> > > > > > issue in jira (https://issues.apache.org/jira/browse/BLUR) and
> > > attach
> > > > > them
> > > > > > there?  Thanks!
> > > > > >
> > > > > > Aaron
> > > > > >
> > > > > > On Tue, May 5, 2015 at 3:42 AM, Ravikumar Govindarajan <
> > > > > > ravikumar.govindarajan@gmail.com <javascript:;>> wrote:
> > > > > >
> > > > > > > Aaron,
> > > > > > >
> > > > > > > I am attaching files for your reference, with minor changes
to
> > > > > > > CacheDirectory. This is on 0.2.2…
> > > > > > >
> > > > > > > Changes look quite simple to plug-in to CacheDirectory
&
> > > > > BlurIndexWarmup.
> > > > > > > Makes me quite suspicious of missing something obvious…
> > > > > > >
> > > > > > > Please do look at it when you find time...
> > > > > > >
> > > > > > > --
> > > > > > > Ravi
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Apr 30, 2015 at 8:25 PM, Aaron McCurry <
> > amccurry@gmail.com
> > > > > <javascript:;>>
> > > > > > wrote:
> > > > > > >
> > > > > > >> Cool, let us know how it goes!
> > > > > > >>
> > > > > > >> Aaron
> > > > > > >>
> > > > > > >> On Thu, Apr 30, 2015 at 3:07 AM, Ravikumar Govindarajan
<
> > > > > > >> ravikumar.govindarajan@gmail.com <javascript:;>>
wrote:
> > > > > > >>
> > > > > > >> > >
> > > > > > >> > > Just so you are aware we are working to allow
Blur to move
> > > > blocks
> > > > > to
> > > > > > >> > > datanodes where the shards are being served
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > Great. I remember that you were telling me this
must be the
> > > > correct
> > > > > > >> > approach for locality sensitive apps on hadoop.
> > > > > > >> >
> > > > > > >> > I am also trying the cache re-load on 0.2.2 itself..
Will
> > update
> > > > you
> > > > > > >> once
> > > > > > >> > it shapes up…
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Ravi
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message