incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <>
Subject [jira] [Commented] (BLUR-132) Create Index Snapshots
Date Thu, 15 Aug 2013 17:49:48 GMT


Aaron McCurry commented on BLUR-132:

I think that you have the right idea but I think it needs to go lower in the system.  I think
that we should add 2 methods to the BlurIndex.

BlurIndex {
  public abstract void createSnapshot(String name) throws IOException;
  public abstract void removeSnapshot(String name) throws IOException;
  public abstract List<String> getSnapshots() throws IOException;

There are 2 sub classes BlurIndexReader and BlurNRTIndex.  For BlurIndexReader, make these
methods throw some sort of unsupported ioexception.  As for BlurNRTIndex you should have everything
you need in that class.  You can control the index deletion policy, you have access to the
writer, etc.

To get this in the system and make it usable, I would first stub out the 3 new methods in
BlurIndex = BlurNRTIndex.  Then modify Blur.thrift to add the three methods, createSnapshot,
removeSnapshot and listSnapshots.

  void createSnapshot(1:string table, 2:string name) throws (1:BlurException ex)
  void removeSnapshot(1:string table, 2:string name) throws (1:BlurException ex)
  // Should return a map where the key is the shard, and the list is the snapshots within
that shard
  map<string,list<string>> listSnapshots(1:string table) throws (1:BlurException

It should probably take the table name and the snapshot name as parameters.  Generate the
thrift code by running in the distribution project.  After that you should
implement the new methods in the BlurShardServer class and the BlurControllerServer class.
 For the shard server, it should get all the indexes from the indexserver and call create/remove/list
on all the indexes and return the results (if any).  The controller server should just spray
the requests to all the shard servers and merge the results.  Take a look at the existing
methods for guidance, and ask if you need any help with that.

After you get the RPC plumbing all hooked up then you can actually call the snapshot code
you provide in the BlurNRTIndex class.  Let me know if you have any more questions.


> Create Index Snapshots
> ----------------------
>                 Key: BLUR-132
>                 URL:
>             Project: Apache Blur
>          Issue Type: New Feature
>    Affects Versions: 0.3.0
>            Reporter: Aaron McCurry
>             Fix For: 0.3.0

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message