hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weichen Ye (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-9426) Make custom distributed barrier procedure pluggable
Date Tue, 26 Aug 2014 11:05:59 GMT

     [ https://issues.apache.org/jira/browse/HBASE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Weichen Ye updated HBASE-9426:
------------------------------

    Description: 

Currently if one wants to implement a custom distributed barrier procedure (e.g., distributed
log roll or distributed table flush), the HBase core code needs to be modified in order for
the procedure to work.

Looking into the snapshot code (especially on region server side), most of the code to enable
the procedure are generic life-cycle management (i.e., init, start, stop). We can make this
part pluggable.

Here is the proposal. Following the coprocessor example, we define two properties:

{code}
hbase.procedure.regionserver.classes
hbase.procedure.master.classes
{code}

The values for both are comma delimited list of classes. On region server side, the classes
implements the following interface:

{code}
public interface RegionServerProcedureManager {
  public void initialize(RegionServerServices rss) throws KeeperException;
  public void start();
  public void stop(boolean force) throws IOException;
  public String getProcedureName();
}
{code}

While on Master side, the classes implement the interface:

{code}
public interface MasterProcedureManager {
  public void initialize(MasterServices master) throws KeeperException, IOException, UnsupportedOperationException;
  public void stop(String why);
  public String getProcedureName();
  public void execProcedure(ProcedureDescription desc) throws IOException;
  IOException;
}
{code}

Where the ProcedureDescription is defined as

{code}
message ProcedureDescription {
  required string name = 1;
  required string instance = 2;
  optional int64 creationTime = 3 [default = 0];
  message Property {
    required string tag = 1;
    optional string value = 2;
  }
  repeated Property props = 4;
}
{code}

A generic API can be defined on HMaster to trigger a procedure:

{code}
public boolean execProcedure(ProcedureDescription desc) throws IOException;
{code}

_SnapshotManager_ and _RegionServerSnapshotManager_ are special examples of _MasterProcedureManager_
and _RegionServerProcedureManager_. They will be automatically included (users don't need
to specify them in the conf file).

  was:
Currently if one wants to implement a custom distributed barrier procedure (e.g., distributed
log roll or distributed table flush), the HBase core code needs to be modified in order for
the procedure to work.

Looking into the snapshot code (especially on region server side), most of the code to enable
the procedure are generic life-cycle management (i.e., init, start, stop). We can make this
part pluggable.

Here is the proposal. Following the coprocessor example, we define two properties:

{code}
hbase.procedure.regionserver.classes
hbase.procedure.master.classes
{code}

The values for both are comma delimited list of classes. On region server side, the classes
implements the following interface:

{code}
public interface RegionServerProcedureManager {
  public void initialize(RegionServerServices rss) throws KeeperException;
  public void start();
  public void stop(boolean force) throws IOException;
  public String getProcedureName();
}
{code}

While on Master side, the classes implement the interface:

{code}
public interface MasterProcedureManager {
  public void initialize(MasterServices master) throws KeeperException, IOException, UnsupportedOperationException;
  public void stop(String why);
  public String getProcedureName();
  public void execProcedure(ProcedureDescription desc) throws IOException;
  IOException;
}
{code}

Where the ProcedureDescription is defined as

{code}
message ProcedureDescription {
  required string name = 1;
  required string instance = 2;
  optional int64 creationTime = 3 [default = 0];
  message Property {
    required string tag = 1;
    optional string value = 2;
  }
  repeated Property props = 4;
}
{code}

A generic API can be defined on HMaster to trigger a procedure:

{code}
public boolean execProcedure(ProcedureDescription desc) throws IOException;
{code}

_SnapshotManager_ and _RegionServerSnapshotManager_ are special examples of _MasterProcedureManager_
and _RegionServerProcedureManager_. They will be automatically included (users don't need
to specify them in the conf file).


> Make custom distributed barrier procedure pluggable 
> ----------------------------------------------------
>
>                 Key: HBASE-9426
>                 URL: https://issues.apache.org/jira/browse/HBASE-9426
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.95.2, 0.94.11
>            Reporter: Richard Ding
>            Assignee: Richard Ding
>             Fix For: 0.99.0
>
>         Attachments: HBASE-9426-4.patch, HBASE-9426-4.patch, HBASE-9426-6.patch, HBASE-9426-7.patch,
HBASE-9426.patch.1, HBASE-9426.patch.2, HBASE-9426.patch.3
>
>
> Currently if one wants to implement a custom distributed barrier procedure (e.g., distributed
log roll or distributed table flush), the HBase core code needs to be modified in order for
the procedure to work.
> Looking into the snapshot code (especially on region server side), most of the code to
enable the procedure are generic life-cycle management (i.e., init, start, stop). We can make
this part pluggable.
> Here is the proposal. Following the coprocessor example, we define two properties:
> {code}
> hbase.procedure.regionserver.classes
> hbase.procedure.master.classes
> {code}
> The values for both are comma delimited list of classes. On region server side, the classes
implements the following interface:
> {code}
> public interface RegionServerProcedureManager {
>   public void initialize(RegionServerServices rss) throws KeeperException;
>   public void start();
>   public void stop(boolean force) throws IOException;
>   public String getProcedureName();
> }
> {code}
> While on Master side, the classes implement the interface:
> {code}
> public interface MasterProcedureManager {
>   public void initialize(MasterServices master) throws KeeperException, IOException,
UnsupportedOperationException;
>   public void stop(String why);
>   public String getProcedureName();
>   public void execProcedure(ProcedureDescription desc) throws IOException;
>   IOException;
> }
> {code}
> Where the ProcedureDescription is defined as
> {code}
> message ProcedureDescription {
>   required string name = 1;
>   required string instance = 2;
>   optional int64 creationTime = 3 [default = 0];
>   message Property {
>     required string tag = 1;
>     optional string value = 2;
>   }
>   repeated Property props = 4;
> }
> {code}
> A generic API can be defined on HMaster to trigger a procedure:
> {code}
> public boolean execProcedure(ProcedureDescription desc) throws IOException;
> {code}
> _SnapshotManager_ and _RegionServerSnapshotManager_ are special examples of _MasterProcedureManager_
and _RegionServerProcedureManager_. They will be automatically included (users don't need
to specify them in the conf file).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message