hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niels Basjes <Ni...@basjes.nl>
Subject Re: Dynamically deploying filters?
Date Sun, 09 Mar 2014 21:44:44 GMT
Thanks for the pointer to HBASE-1936.
Very interesting feature.


On Sun, Mar 9, 2014 at 8:03 PM, Bharath Vissapragada
<bharathv@cloudera.com>wrote:

> Hey Niels,
>
> Did you go through HBASE-1936? You can just upload the jars to a path in
> hdfs and not to all regionservers.
>
> IIRC, "refreshing" a jar doesn't work. That means you need add jars with
> new names and also since the code uses URLClassloader, classes once loaded
> cannot be unloaded since the current loader still has references to it.
>
> So you need to maintain different versions of your filter incase you are
> using a modified version to test. I believe you can use this for your
> testing. All you need to do is to make sure the new version of filters have
> new classnames and modify your testing code accordingly.
>
>
>
>
>
>
> On Sun, Mar 9, 2014 at 9:14 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > The original blog didn't mention security.
> >
> > If I understand correctly, the application of custom filters is after ACL
> > check in a secure cluster.
> > The cell visibility feature in 0.98 is implemented through
> > VisibilityController which builds on top of BaseRegionObserver.
> >
> > So we should be fine.
> >
> >
> > On Sun, Mar 9, 2014 at 12:39 AM, Niels Basjes <Niels@basjes.nl> wrote:
> >
> > > From what I see this is not putting those classes on into the cluster
> at
> > > all. This looks like it is serializing them during each scan.
> > > So this issue does not arise.
> > > What I'm thinking about is how to ensure that this is not a way to
> avoid
> > > the security in the cluster.
> > >
> > > Niels
> > > On Mar 7, 2014 1:41 AM, "Ted Yu" <yuzhihong@gmail.com> wrote:
> > >
> > > > Interesting blog.
> > > >
> > > > I wonder how subsequent work addresses the following:
> > > >
> > > > bq. Updating the filter.jar in the Hadoop FS while a table scan is
> > > > happening can have undesired results if the updated filters are not
> > > > backward compatible.
> > > >
> > > >
> > > > On Thu, Mar 6, 2014 at 12:54 PM, Niels Basjes <Niels@basjes.nl>
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > In the current HBase versions a Filter needs to be deployed by
> > putting
> > > a
> > > > > jar into all region servers (and depending on the HBase version
> > restart
> > > > the
> > > > > regionservers).
> > > > >
> > > > > I'm in a multi tenant cluster environment where we may run into the
> > > need
> > > > to
> > > > > have both the old and the new version of a Filter available at the
> > same
> > > > > time. Also the option of having a method of easily trying out a new
> > > > > implementation for a Filter (to see if it performs better) would
> be a
> > > lot
> > > > > easier if it were possible to use a custom Filter without having
to
> > put
> > > > it
> > > > > onto all region servers.
> > > > >
> > > > > So after some Googling I found this interesting experiment for
> > > > dynamically
> > > > > uploading the Filter code with the Scan:
> > > > >
> > > > >
> > > >
> > >
> >
> http://tech.flurry.com/2012/12/06/exploring-dynamic-loading-of-custom-filters-i/
> > > > >
> > > > >
> > > > > My question: Is such a feature planned for the mainline HBase?
> > > > >
> > > > > --
> > > > > Best regards
> > > > >
> > > > > Niels Basjes
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Bharath Vissapragada
> <http://www.cloudera.com>
>



-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message