accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Vines" <vi...@apache.org>
Subject Re: Review Request 23397: ACCUMULO-2984 Support Running MAC against a standard instance
Date Tue, 21 Oct 2014 00:32:25 GMT


> On Oct. 20, 2014, 9:02 p.m., kturner wrote:
> > minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java,
line 471
> > <https://reviews.apache.org/r/23397/diff/7/?file=726322#file726322line471>
> >
> >     In the IT I wrote, this code is causing problems.  The following situation is
occuring.
> >     
> >      * Create MAC to connection to Accumulo instance 1
> >      * Stop ZK1 for Accumulo instance 1
> >      * Create MAC to connection to Accumulo instance 2 (Accumulo instance 2 uses
ZK2)
> >     
> >     I think the code gets stuck trying to connect to ZK1, when trying to connection
to Accumulo instance 2
> 
> John Vines wrote:
>     So you're expecting this to work with a new zookeeper? Standard Accumulo doesn't
work with a new ZK instance, why would you expect the MAC equivalent to?
> 
> kturner wrote:
>     Its just what my IT did.  Although I think it makes sense to fix this (could be a
follow on issue).  Unless this is fixed, this new functionality can only be used to run one
Accumulo instance per Java process(or classloader).  It may also cache instance name??? not
sure.   It would be nice to use this functionallity to run existing Accumulo instance A and
then run existing Accumulo instance B from the same process.
> 
> kturner wrote:
>     > Standard Accumulo doesn't work with a new ZK instance, why would you expect
the MAC equivalent to?
>     
>     To clarify, my IT created ZK1 and Accumulo instance 1 in testA.  Then testB created
ZK2 and instance 2 in the same process.  When I tried to run this second instance it hung.
 I think it was trying to connect to the old ZK.   So I am not using a new ZK with an existing
Accumulo instance.
> 
> kturner wrote:
>     I outlined the situation incorrectly.  Should have said the following.
>     
>     
>      * Create MAC1 to run Accumulo instance 1
>      * Stop MAC1
>      * Stop ZK1 for Accumulo instance 1
>      * Create MAC2 to run Accumulo instance 2 (Accumulo instance 2 uses ZK2)

The short explanation here is that this is using singleton items for parsing some information.
Specifically, this is HdfsZooinstance, ZooReaderWriter, etc. I have resolved these issues
in my latest patch, but I'm not sure how welcoming people would be to added methods for clearing
out singleton instances.


- John


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23397/#review57431
-----------------------------------------------------------


On Oct. 21, 2014, 12:32 a.m., John Vines wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23397/
> -----------------------------------------------------------
> 
> (Updated Oct. 21, 2014, 12:32 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-2984
>     https://issues.apache.org/jira/browse/ACCUMULO-2984
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> Adds a change to SiteConfiguration to allow external setting of the xml configuration
file.
> Adds a single method to MiniAccumuloConfig which allows a user to point to accumulo-site.xml
and HADOOP_CONF_DIR to use for pulling out requisite instance information
> Clusters configurations into those required to run inside a MAC-sized footprint and those
which are for arbitrary naming schemes for MAC
> Provides flagging to prevent uneccessary folder creation
> Provides flagging to prevent running zookeeper and initializing
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java 9b65e7d 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java be80f85

>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java
5d8501e 
>   minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java
e9ad045 
>   server/base/src/main/java/org/apache/accumulo/server/client/HdfsZooInstance.java 3508164

>   server/base/src/main/java/org/apache/accumulo/server/conf/ServerConfiguration.java
50dec57 
>   server/base/src/main/java/org/apache/accumulo/server/util/AccumuloStatus.java PRE-CREATION

>   server/base/src/main/java/org/apache/accumulo/server/zookeeper/ZooReaderWriter.java
435591d 
>   test/src/test/java/org/apache/accumulo/test/ExistingMacIT.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/23397/diff/
> 
> 
> Testing
> -------
> 
> Ran the following test code-
> public class TestMACWithRealInstance {
>   public static void main(String args[]) throws IOException, AccumuloException, AccumuloSecurityException,
TableExistsException, InterruptedException {
>     MiniAccumuloConfig macConfig = new MiniAccumuloConfig(new File("/tmp/mac"), "secret");
>     macConfig.setNumTservers(2);
>     macConfig.setMemory(ServerType.TABLET_SERVER, 2, MemoryUnit.GIGABYTE);
>     macConfig.useExistingInstance(new File("/usr/lib/accumulo/conf/accumulo-site.xml"),
new File("/usr/lib/hadoop/conf"));
>     MiniAccumuloCluster mac = new MiniAccumuloCluster(macConfig);
>     mac.start();
>     System.out.println("Started");
>     mac.getConnector("root", "secret").tableOperations().create("macCreated");
>     System.out.println("Stopping");
>     mac.stop();
>     System.out.println("Stopped");
>   }
> }
> Which runs fine, except stopping issues which seem to be related to ACCUMULO-2985
> 
> After running this, I validated that the table was created in the real accumulo instance
via zkCli
> 
> 
> Thanks,
> 
> John Vines
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message