drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4199) Add Support for HBase 1.X
Date Fri, 08 Apr 2016 02:45:25 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231538#comment-15231538
] 

ASF GitHub Bot commented on DRILL-4199:
---------------------------------------

Github user jacques-n commented on a diff in the pull request:

    https://github.com/apache/drill/pull/443#discussion_r58974382
  
    --- Diff: contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePlugin.java
---
    @@ -19,36 +19,75 @@
     
     import java.io.IOException;
     import java.util.Set;
    +import java.util.concurrent.ExecutionException;
    +import java.util.concurrent.TimeUnit;
     
     import org.apache.calcite.schema.SchemaPlus;
    -
     import org.apache.drill.common.JSONOptions;
    +import org.apache.drill.common.exceptions.UserException;
     import org.apache.drill.exec.ops.OptimizerRulesContext;
     import org.apache.drill.exec.server.DrillbitContext;
     import org.apache.drill.exec.store.AbstractStoragePlugin;
     import org.apache.drill.exec.store.SchemaConfig;
     import org.apache.drill.exec.store.StoragePluginOptimizerRule;
    +import org.apache.hadoop.hbase.client.Connection;
    +import org.apache.hadoop.hbase.client.ConnectionFactory;
     
     import com.fasterxml.jackson.core.type.TypeReference;
     import com.fasterxml.jackson.databind.ObjectMapper;
    +import com.google.common.cache.CacheBuilder;
    +import com.google.common.cache.CacheLoader;
    +import com.google.common.cache.LoadingCache;
    +import com.google.common.cache.RemovalListener;
    +import com.google.common.cache.RemovalNotification;
     import com.google.common.collect.ImmutableSet;
    +import com.google.common.util.concurrent.UncheckedExecutionException;
     
     public class HBaseStoragePlugin extends AbstractStoragePlugin {
    -  static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(HBaseStoragePlugin.class);
    +  private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(HBaseStoragePlugin.class);
     
       private final DrillbitContext context;
    -  private final HBaseStoragePluginConfig engineConfig;
    +  private final HBaseStoragePluginConfig storeConfig;
       private final HBaseSchemaFactory schemaFactory;
     
       @SuppressWarnings("unused")
       private final String name;
     
    -  public HBaseStoragePlugin(HBaseStoragePluginConfig configuration, DrillbitContext context,
String name)
    +  private final LoadingCache<Integer, Connection> connectionCache;
    +
    +  public HBaseStoragePlugin(HBaseStoragePluginConfig storeConfig, DrillbitContext context,
String name)
           throws IOException {
         this.context = context;
         this.schemaFactory = new HBaseSchemaFactory(this, name);
    -    this.engineConfig = configuration;
    +    this.storeConfig = storeConfig;
         this.name = name;
    +
    +    this.connectionCache = CacheBuilder.newBuilder()
    +        .expireAfterAccess(24, TimeUnit.HOURS)
    +        .removalListener(new RemovalListener<Integer, Connection>() {
    +          @Override
    +          public void onRemoval(RemovalNotification<Integer, Connection> notification)
{
    +            try {
    +              notification.getValue().close();
    +              logger.debug("HBase connection '{}' closed.", notification.getValue());
    +            } catch (Throwable t) {
    +              logger.warn("Error while closing HBase connection.", t);
    +            }
    +          }
    +        })
    +        .build(new CacheLoader<Integer, Connection>() {
    +          @Override
    +          public synchronized Connection load(Integer key /* ignored */) throws Exception
{
    +            if (connection == null
    +                || connection.isAborted()
    +                || connection.isClosed()) {
    +              connection = ConnectionFactory.createConnection(HBaseStoragePlugin.this.storeConfig.getHBaseConf());
    +              logger.debug("HBase connection '{}' created.", connection);
    +            }
    +            return connection;
    +          }
    +          private Connection connection = null;
    --- End diff --
    
    Why is this an instance field and not protected? Since the cache loader is already managing
a single instance, it doesn't seem like we need to worry about ensuring a singleton.  Separately,
since we're behind a cache, we won't call connection.isAborted(). Should we make sure that
the get() checks isAborted() and isClosed()


> Add Support for HBase 1.X
> -------------------------
>
>                 Key: DRILL-4199
>                 URL: https://issues.apache.org/jira/browse/DRILL-4199
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Storage - HBase
>    Affects Versions: 1.7.0
>            Reporter: Divjot singh
>            Assignee: Aditya Kishore
>
> Is there any Road map to upgrade the Hbase version to 1.x series. Currently drill supports
Hbase 0.98 version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message