incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <>
Subject [jira] [Commented] (BLUR-112) Allow for types to be set on blur tables
Date Tue, 25 Jun 2013 05:46:20 GMT


Rahul Challapalli commented on BLUR-112:

The below observations are based on the assumption that we will update AnalyzerDefinition
with a new ColumnDefinition when dynamic columns are added. Correct me if thats not the case.

Once defined, we do not allow updates for existing column definitions. We can safely check
whether any Inbound Column types have a conflict with any defined types in AnalyzerDefinition
as we do not have to worry about AnalyzerDefinition object on the shard server being out of
sync with that stored on Zookeeper. 
However for dynamic columns as you explained, the same column might have been added by a different
shard server. This need not happen at the exact same time(correct me) . Even if ShardServer-B
added a dynamic column 5 seconds before, the AnalyzerDefinition in ShardServer-A does not
reflect that change

Also a little confused about how to use ZkCachedMap. What values will we be caching/storing
using this? Are they only the one's which can be overwritten on Zookeeper?

Have a look at the below pseudo code and let me know what you think

  BlurAnalyzer analyzer = _indexServer.getAnalyzer(table);
  AnalyzerDefinition analyzerDefinition = analyzer.getAnalyzerDefinition();
  if (!analyzer.containsColumn( {
    // add new ColumnDefinition to AnalyzerDefinition
  // reload the BlurAnalyzer(update _typeLookup and _fieldTypes)
  String analyzerJsonString = analyzer.toJSON();
  String blurTablePath = ZookeeperPathConstants.getTablePath(_cluster, table);
  if (_zkCachedMap.putIfMissing(blurTablePath, analyzerJsonString)) {
    // successfull
  } else {
    // load analyzer definition from Zookeeper and then reload BlurAnalyzer
    throw new BlurException("Column already defined");
> Allow for types to be set on blur tables
> ----------------------------------------
>                 Key: BLUR-112
>                 URL:
>             Project: Apache Blur
>          Issue Type: Improvement
>    Affects Versions: 0.2.0, 0.3.0
>            Reporter: Aaron McCurry
>             Fix For: 0.3.0
> Create the ability for Blur to handle the default Lucene field types.  This should not
be tied to the table descriptor because types should be allowed to be added at runtime.  Also
2 new fields should be added to the TableDescriptor:
> 1. A strict types attribute.  If set to true, if a new column is added to the table and
there is no type mapping for it.  Throw an exception.  Set to false by default.
> 2. Default type is strict is set to false.  The default type should be text.
> Also, dynamic columns could be allowed if their name included the type.  Such as:
> The column name could be "col1" with a type of "int", in the Column struct in thrift
the name would be "col1/int" and if the type did not exist before the call it would be added.
> Thoughts?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message