Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Content-Type: multipart/alternative;
 boundary="===============1340857426249454074=="
MIME-Version: 1.0
Subject: Re: Review Request 23397: ACCUMULO-2984 Support Running MAC against a
 standard instance
From: keith@deenlo.com
To: "John Vines" <vines@apache.org>, "accumulo" <dev@accumulo.apache.org>,
 keith@deenlo.com
Date: Tue, 21 Oct 2014 15:09:53 -0000
Message-ID: <20141021150953.1283.44566@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: noreply@reviews.apache.org
References: <20141021003212.1282.33830@reviews.apache.org>
In-Reply-To: <20141021003212.1282.33830@reviews.apache.org>
Reply-To: keith@deenlo.com

--===============1340857426249454074==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23397/#review57582
-----------------------------------------------------------


minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java
<https://reviews.apache.org/r/23397/#comment98367>

    I looked into this code, its full of landmines.  It calls VolumeManager.get() which uses static config for accumulo-site.xml (which may not be the accumulo-site you want).   Want to avoid this.  Want to make sure that all info is obtained using your accumulo-site.xml and your haddop config.
    
    Below I put together some code that I think mostly does this.  Tried to make sure the instance id is obtained from filesystem using your config and that zookeeper is obtained using your config.
    
    ```java
    Configuration config = new Configuration(false);
    config.addResource(yourAccumuloSite);
    
    ConfigurationCopy cc = new ConfigurationCopy(config);
    try {
      fs = VolumeManagerImpl.get(cc);
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
    Path instanceIdPath = Accumulo.getAccumuloInstanceIdPath(fs);
    
    //TODO this method calls CahcedConfig.getInstance(), should create a method where hadoop config is passed in
    String instanceIdFromFile = ZooUtil.getInstanceIDFromHdfs(instanceIdPath, cc);
    
    zrw = new ZooReaderWriterFactory.getZooReaderWriter(cc.get(Property.INSTANCE_ZK_HOST), (int) cc.getTimeInMillis(Property.INSTANCE_ZK_TIMEOUT),
                cc.get(Property.INSTANCE_SECRET));
    
    String rootPath = ZooUtil.getRoot(instanceIdFromFile);
    //TODO pass this root path to isAccumuloOffline
    
    ```


- kturner


On Oct. 21, 2014, 12:32 a.m., John Vines wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23397/
> -----------------------------------------------------------
> 
> (Updated Oct. 21, 2014, 12:32 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-2984
>     https://issues.apache.org/jira/browse/ACCUMULO-2984
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> Adds a change to SiteConfiguration to allow external setting of the xml configuration file.
> Adds a single method to MiniAccumuloConfig which allows a user to point to accumulo-site.xml and HADOOP_CONF_DIR to use for pulling out requisite instance information
> Clusters configurations into those required to run inside a MAC-sized footprint and those which are for arbitrary naming schemes for MAC
> Provides flagging to prevent uneccessary folder creation
> Provides flagging to prevent running zookeeper and initializing
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java 9b65e7d 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java be80f85 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java 5d8501e 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java e9ad045 
>   server/base/src/main/java/org/apache/accumulo/server/client/HdfsZooInstance.java 3508164 
>   server/base/src/main/java/org/apache/accumulo/server/conf/ServerConfiguration.java 50dec57 
>   server/base/src/main/java/org/apache/accumulo/server/util/AccumuloStatus.java PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/zookeeper/ZooReaderWriter.java 435591d 
>   test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/23397/diff/
> 
> 
> Testing
> -------
> 
> Ran the following test code-
> public class TestMACWithRealInstance {
>   public static void main(String args[]) throws IOException, AccumuloException, AccumuloSecurityException, TableExistsException, InterruptedException {
>     MiniAccumuloConfig macConfig = new MiniAccumuloConfig(new File("/tmp/mac"), "secret");
>     macConfig.setNumTservers(2);
>     macConfig.setMemory(ServerType.TABLET_SERVER, 2, MemoryUnit.GIGABYTE);
>     macConfig.useExistingInstance(new File("/usr/lib/accumulo/conf/accumulo-site.xml"), new File("/usr/lib/hadoop/conf"));
>     MiniAccumuloCluster mac = new MiniAccumuloCluster(macConfig);
>     mac.start();
>     System.out.println("Started");
>     mac.getConnector("root", "secret").tableOperations().create("macCreated");
>     System.out.println("Stopping");
>     mac.stop();
>     System.out.println("Stopped");
>   }
> }
> Which runs fine, except stopping issues which seem to be related to ACCUMULO-2985
> 
> After running this, I validated that the table was created in the real accumulo instance via zkCli
> 
> 
> Thanks,
> 
> John Vines
> 
>


--===============1340857426249454074==--