incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <>
Subject [jira] [Commented] (BLUR-251) Create table level third party field types
Date Tue, 22 Oct 2013 11:38:43 GMT


Aaron McCurry commented on BLUR-251:

I think that if someone is going to add a new type into Blur that we can assume that they
are fairly technical.  So I would take the approach that they can place the jar file(s) somewhere
in HDFS to reference.  Also I don't we should assume they only have a single jar file.

The best model to follow on this one is HBase's coprocessors, they even have a classloader
that we can use.  So I would change the method sigs above to be a single one.  If you need
a system level type I think we can have people install it via the properties.

public void addCustomFieldTypeToTable(String tablename, String fieldTypeClassName, String

Where the uriLocation can be a file (jar file) or a directory of files to be accessed by the
classloader.  Since the name of the type is defined in code, we really only need the classname
of the type to be loaded.

Now to actually make it accessible in the system you will need to modify the properties of
the table and save back into zookeeper.  Should probably do this in the ClusterStatus object,
make it so you can append new properties but not change any existing.  After that has happened,
you need to call TableContext.clear() on all the servers, this should probably be done via
a ZooKeeper watcher of some sort.

I can help point you to the locations of all the different components if you have questions.


> Create table level third party field types
> ------------------------------------------
>                 Key: BLUR-251
>                 URL:
>             Project: Apache Blur
>          Issue Type: Sub-task
>          Components: Blur
>    Affects Versions: 0.3.0, 0.2.1
>            Reporter: Aaron McCurry
>             Fix For: 0.3.0
> Allow for dynamically adding new field types to a given table.  Each type will have it's
own class loader and will load the jar files from a location within the table's storage directory.
 When the type is created the jars files will be copied from their location into the table
storage so that they can live with the table and are effectively immutable.  This will require
new methods in the Blur Thrift API.

This message was sent by Atlassian JIRA

View raw message