Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 16 Feb 2016 02:17:18 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12931522.1452879121000.56977.1455589038136@Atlassian.JIRA>
In-Reply-To: <JIRA.12931522.1452879121000@Atlassian.JIRA>
References: <JIRA.12931522.1452879121000@Atlassian.JIRA>
 <JIRA.12931522.1452879121872@arcas>
Subject: [jira] [Commented] (DRILL-4275) Refactor e/pstore interfaces and
 their factories to provide a unified mechanism to access stores
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DRILL-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148016#comment-15148016 ] 

ASF GitHub Bot commented on DRILL-4275:
---------------------------------------

Github user hnfgns commented on a diff in the pull request:

    https://github.com/apache/drill/pull/374#discussion_r52962256
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/store/provider/ZookeeperPersistentStoreProvider.java ---
    @@ -0,0 +1,85 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.drill.exec.store.sys.store.provider;
    +
    +import java.io.IOException;
    +
    +import com.google.common.annotations.VisibleForTesting;
    +import org.apache.curator.framework.CuratorFramework;
    +import org.apache.drill.common.config.DrillConfig;
    +import org.apache.drill.exec.coord.zk.ZKClusterCoordinator;
    +import org.apache.drill.exec.exception.StoreException;
    +import org.apache.drill.exec.store.dfs.DrillFileSystem;
    +import org.apache.drill.exec.store.sys.PersistentStore;
    +import org.apache.drill.exec.store.sys.PersistentStoreRegistry;
    +import org.apache.drill.exec.store.sys.PersistentStoreConfig;
    +import org.apache.drill.exec.store.sys.store.LocalPersistentStore;
    +import org.apache.drill.exec.store.sys.store.ZookeeperPersistentStore;
    +import org.apache.hadoop.fs.Path;
    +import org.slf4j.Logger;
    +import org.slf4j.LoggerFactory;
    +
    +public class ZookeeperPersistentStoreProvider extends BasePersistentStoreProvider {
    +  private static final Logger logger = LoggerFactory.getLogger(ZookeeperPersistentStoreProvider.class);
    +
    +  private static final String DRILL_EXEC_SYS_STORE_PROVIDER_ZK_BLOBROOT = "drill.exec.sys.store.provider.zk.blobroot";
    +
    +  private final CuratorFramework curator;
    +  private final DrillFileSystem fs;
    +  private final Path blobRoot;
    +
    +  public ZookeeperPersistentStoreProvider(final PersistentStoreRegistry<ZKClusterCoordinator> registry) throws StoreException {
    +    this(registry.getConfig(), registry.getCoordinator().getCurator());
    +  }
    +
    +  @VisibleForTesting
    +  public ZookeeperPersistentStoreProvider(final DrillConfig config, final CuratorFramework curator) throws StoreException {
    +    this.curator = curator;
    +
    +    if (config.hasPath(DRILL_EXEC_SYS_STORE_PROVIDER_ZK_BLOBROOT)) {
    +      blobRoot = new Path(config.getString(DRILL_EXEC_SYS_STORE_PROVIDER_ZK_BLOBROOT));
    +    }else{
    +      blobRoot = LocalPersistentStore.getLogDir();
    +    }
    +
    +    try {
    +      this.fs = LocalPersistentStore.getFileSystem(config, blobRoot);
    +    } catch (IOException ex) {
    +      throw new StoreException("unable to get filesystem", ex);
    +    }
    +  }
    +
    +  @Override
    +  public <V> PersistentStore<V> getOrCreateStore(final PersistentStoreConfig<V> config) throws StoreException {
    +    switch(config.getMode()){
    +    case BLOB_PERSISTENT:
    +      return new LocalPersistentStore<>(fs, blobRoot, config);
    --- End diff --
    
    zk relays blob persistency to fs.


> Refactor e/pstore interfaces and their factories to provide a unified mechanism to access stores
> ------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4275
>                 URL: https://issues.apache.org/jira/browse/DRILL-4275
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Flow
>            Reporter: Hanifi Gunes
>            Assignee: Deneche A. Hakim
>
> We rely on E/PStore interfaces to persist data. Even though E/PStore stands for Ephemeral and Persistent stores respectively, the current design for EStore does not extend the interface/functionality of PStore at all, which hints abstraction for EStore is redundant. This issue proposes a new unified Store interface replacing the old E/PStore that exposes an additional method that report persistence level as follows:
> {code:title=Store interface}
> interface Store<V> {
>   StoreMode getMode();
>   V get(String key);
>   ...
> }
> enum StoreMode {
>   EPHEMERAL,
>   PERSISTENT,
>   ...
> }
> {code}
> The new design brings in less redundancy, more centralized code, ease to reason and maintain.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)