accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vi...@apache.org
Subject svn commit: r1499116 [1/2] - in /accumulo/branches/ACCUMULO-CURATOR: ./ core/ core/src/main/java/org/apache/accumulo/core/client/impl/ core/src/main/java/org/apache/accumulo/core/conf/ examples/ examples/simple/src/main/java/org/apache/accumulo/example...
Date Tue, 02 Jul 2013 21:25:59 GMT
Author: vines
Date: Tue Jul  2 21:25:56 2013
New Revision: 1499116

URL: http://svn.apache.org/r1499116
Log:
Merging up to trunk



Added:
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/
      - copied from r1499098, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/FateStarvationIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/GarbageCollectorIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/GarbageCollectorIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/LogicalTimeIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/MapReduceIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/MapReduceIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/MaxOpenIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/MergeIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/MergeMetaIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/MergeMetaIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/PermissionsIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/RowDeleteIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/ScanIteratorIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/ScanRangeIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/ScanSessionTimeOutIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/ServerSideErrorIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/ShutdownIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/ShutdownIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/SimpleBalancerFairnessIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/SimpleBalancerFairnessIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/SparseColumnFamilyIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/SplitIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/StartIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/StartIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/TableIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/TableIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/TabletIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java
      - copied unchanged from r1499112, accumulo/trunk/test/src/test/java/org/apache/accumulo/test/functional/TimeoutIT.java
Removed:
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/GCLotsOfCandidatesTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/AddSplitTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/BadIteratorMincTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/BatchScanSplitTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/BatchWriterFlushTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/BloomFilterTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/BulkFileTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/BulkSplitOptimizationTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/ConcurrencyTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/ConstraintTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/CreateAndUseTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/DeleteEverythingTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/DeleteRowsSplitTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/DeleteRowsTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/FateStarvationTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/LargeRowTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/LogicalTimeTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/MaxOpenTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/MergeTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/PermissionsTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/RowDeleteTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/ScanIteratorTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/ScanRangeTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/ScanSessionTimeOutTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/ServerSideErrorTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/SparseColumnFamilyTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/TimeoutTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/addSplit.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/aggregation.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/baditerminc.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/batchScanSplit.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/batchWriterFlush.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/binary.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/bloom.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/bulk.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/bulkFile.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/bulkSplitOptimization.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/combiner.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/compaction.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/concurrency.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/constraint.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/createAndUse.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/delete.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/deleteEverything.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/deleterows.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/dynamic.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/dynamicThreadPools.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/examples.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/fateStartvation.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/gc.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/largeRow.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/logicalTime.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/mapreduce.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/maxOpen.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/merge.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/mergeMetadata.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/range.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/readwrite.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/rename.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/rowDelete.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/scanIter.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/scanSessionTimeout.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/security.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/serverSideError.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/shutdown.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/simpleBalancer.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/sparseColumnFamily.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/split.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/start.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/table.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/tablets.py
    accumulo/branches/ACCUMULO-CURATOR/test/system/auto/simple/timeout.py
Modified:
    accumulo/branches/ACCUMULO-CURATOR/   (props changed)
    accumulo/branches/ACCUMULO-CURATOR/core/   (props changed)
    accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchReader.java
    accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java
    accumulo/branches/ACCUMULO-CURATOR/examples/   (props changed)
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/dirlist/QueryUtil.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/helloworld/ReadData.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/RowHash.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TableToFile.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/WordCount.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/bulk/VerifyIngest.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Index.java
    accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Query.java
    accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java
    accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java
    accumulo/branches/ACCUMULO-CURATOR/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java
    accumulo/branches/ACCUMULO-CURATOR/server/   (props changed)
    accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/master/recovery/HadoopLogCloser.java
    accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/util/CheckForMetadataProblems.java
    accumulo/branches/ACCUMULO-CURATOR/test/pom.xml
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestBinaryRows.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestIngest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/VerifyIngest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/functional/SlowIterator.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/performance/metadata/MetadataBatchScanTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/AuditMessageTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/MetaSplitTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/ShellServerTest.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/TestAccumuloSplitRecovery.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/DeleteIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/DynamicThreadPoolsIT.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/FunctionalTestUtils.java
    accumulo/branches/ACCUMULO-CURATOR/test/src/test/java/org/apache/accumulo/test/functional/ReadWriteIT.java

Propchange: accumulo/branches/ACCUMULO-CURATOR/
------------------------------------------------------------------------------
  Merged /accumulo/trunk:r1496191-1499112

Propchange: accumulo/branches/ACCUMULO-CURATOR/core/
------------------------------------------------------------------------------
  Merged /accumulo/trunk/core:r1496191-1499112

Modified: accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchReader.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchReader.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchReader.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchReader.java Tue Jul  2 21:25:56 2013
@@ -45,6 +45,7 @@ public class TabletServerBatchReader ext
   
   private TCredentials credentials;
   private Authorizations authorizations = Authorizations.EMPTY;
+  private Throwable ex = null;
   
   private static int nextBatchReaderInstance = 1;
   
@@ -65,6 +66,7 @@ public class TabletServerBatchReader ext
     queryThreadPool = new SimpleThreadPool(numQueryThreads, "batch scanner " + batchReaderInstance + "-");
     
     ranges = null;
+    ex = new Throwable();
   }
   
   @Override
@@ -78,7 +80,7 @@ public class TabletServerBatchReader ext
   @Override
   protected void finalize() {
     if (!queryThreadPool.isShutdown()) {
-      log.warn(TabletServerBatchReader.class.getSimpleName() + " not shutdown; did you forget to call close()?");
+      log.warn(TabletServerBatchReader.class.getSimpleName() + " not shutdown; did you forget to call close()?", ex);
       close();
     }
   }

Modified: accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/core/src/main/java/org/apache/accumulo/core/conf/SiteConfiguration.java Tue Jul  2 21:25:56 2013
@@ -49,7 +49,7 @@ public class SiteConfiguration extends A
       xmlConfig = new Configuration(false);
       
       if (SiteConfiguration.class.getClassLoader().getResource(configFile) == null)
-        log.warn(configFile + " not found on classpath");
+        log.warn(configFile + " not found on classpath", new Throwable());
       else
         xmlConfig.addResource(configFile);
     }

Propchange: accumulo/branches/ACCUMULO-CURATOR/examples/
------------------------------------------------------------------------------
  Merged /accumulo/trunk/examples:r1496191-1499098

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/dirlist/QueryUtil.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/dirlist/QueryUtil.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/dirlist/QueryUtil.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/dirlist/QueryUtil.java Tue Jul  2 21:25:56 2013
@@ -256,7 +256,7 @@ public class QueryUtil {
   public static class Opts extends ClientOnRequiredTable {
     @Parameter(names="--path", description="the directory to list")
     String path = "/";
-    @Parameter(names="--search", description="find a file or directorys with the given name")
+    @Parameter(names="--search", description="find a file or directory with the given name")
     boolean search = false;
   }
   

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/helloworld/ReadData.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/helloworld/ReadData.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/helloworld/ReadData.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/helloworld/ReadData.java Tue Jul  2 21:25:56 2013
@@ -30,6 +30,7 @@ import org.apache.accumulo.core.data.Key
 import org.apache.accumulo.core.data.Range;
 import org.apache.accumulo.core.data.Value;
 import org.apache.hadoop.io.Text;
+import org.apache.log4j.Logger;
 
 import com.beust.jcommander.Parameter;
 
@@ -38,13 +39,15 @@ import com.beust.jcommander.Parameter;
  */
 public class ReadData {
   
+  private static final Logger log = Logger.getLogger(ReadData.class);
+  
   static class Opts extends ClientOnRequiredTable {
     @Parameter(names="--startKey")
     String startKey;
     @Parameter(names="--endKey")
     String endKey;
   }
-  
+
   public static void main(String[] args) throws AccumuloException, AccumuloSecurityException, TableNotFoundException {
     Opts opts = new Opts();
     ScannerOpts scanOpts = new ScannerOpts();
@@ -67,8 +70,8 @@ public class ReadData {
       Entry<Key,Value> e = iter.next();
       Text colf = e.getKey().getColumnFamily();
       Text colq = e.getKey().getColumnQualifier();
-      System.out.print("row: " + e.getKey().getRow() + ", colf: " + colf + ", colq: " + colq);
-      System.out.println(", value: " + e.getValue().toString());
+      log.trace("row: " + e.getKey().getRow() + ", colf: " + colf + ", colq: " + colq);
+      log.trace(", value: " + e.getValue().toString());
     }
   }
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/RowHash.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/RowHash.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/RowHash.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/RowHash.java Tue Jul  2 21:25:56 2013
@@ -89,8 +89,6 @@ public class RowHash extends Configured 
   }
   
   public static void main(String[] args) throws Exception {
-    int res = ToolRunner.run(CachedConfiguration.getInstance(), new RowHash(), args);
-    if (res != 0)
-      System.exit(res);
+    ToolRunner.run(CachedConfiguration.getInstance(), new RowHash(), args);
   }
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TableToFile.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TableToFile.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TableToFile.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TableToFile.java Tue Jul  2 21:25:56 2013
@@ -50,7 +50,7 @@ public class TableToFile extends Configu
     @Parameter(names = "--output", description = "output directory", required = true)
     String output;
     @Parameter(names = "--columns", description = "columns to extract, in cf:cq{,cf:cq,...} form")
-    String columns;
+    String columns = "";
   }
   
   /**
@@ -123,8 +123,6 @@ public class TableToFile extends Configu
    * @throws Exception
    */
   public static void main(String[] args) throws Exception {
-    int res = ToolRunner.run(CachedConfiguration.getInstance(), new TableToFile(), args);
-    if (res != 0)
-      System.exit(res);
+    ToolRunner.run(CachedConfiguration.getInstance(), new TableToFile(), args);
   }
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java Tue Jul  2 21:25:56 2013
@@ -349,8 +349,7 @@ public class TeraSortIngest extends Conf
   }
   
   public static void main(String[] args) throws Exception {
-    int res = ToolRunner.run(CachedConfiguration.getInstance(), new TeraSortIngest(), args);
-    System.exit(res);
+    ToolRunner.run(CachedConfiguration.getInstance(), new TeraSortIngest(), args);
   }
   
   static class Opts extends ClientOnRequiredTable {

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/WordCount.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/WordCount.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/WordCount.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/WordCount.java Tue Jul  2 21:25:56 2013
@@ -88,7 +88,6 @@ public class WordCount extends Configure
   }
   
   public static void main(String[] args) throws Exception {
-    int res = ToolRunner.run(CachedConfiguration.getInstance(), new WordCount(), args);
-    System.exit(res);
+    ToolRunner.run(CachedConfiguration.getInstance(), new WordCount(), args);
   }
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/bulk/VerifyIngest.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/bulk/VerifyIngest.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/bulk/VerifyIngest.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/bulk/VerifyIngest.java Tue Jul  2 21:25:56 2013
@@ -81,8 +81,6 @@ public class VerifyIngest {
     
     if (ok)
       System.out.println("OK");
-    
-    System.exit(ok ? 0 : 1);
   }
   
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Index.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Index.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Index.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Index.java Tue Jul  2 21:25:56 2013
@@ -68,8 +68,7 @@ public class Index {
       bw.addMutation(m);
   }
   
-  private static void index(int numPartitions, File src, String splitRegex, BatchWriter bw) throws Exception {
-    
+  public static void index(int numPartitions, File src, String splitRegex, BatchWriter bw) throws Exception {
     if (src.isDirectory()) {
       for (File child : src.listFiles()) {
         index(numPartitions, child, splitRegex, bw);

Modified: accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Query.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Query.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Query.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/examples/simple/src/main/java/org/apache/accumulo/examples/simple/shard/Query.java Tue Jul  2 21:25:56 2013
@@ -48,31 +48,37 @@ public class Query {
     List<String> terms = new ArrayList<String>();
   }
   
-  /**
-   * @param args
-   */
-  public static void main(String[] args) throws Exception {
-    Opts opts = new Opts();
-    BatchScannerOpts bsOpts = new BatchScannerOpts();
-    opts.parseArgs(Query.class.getName(), args, bsOpts);
-    
-    Connector conn = opts.getConnector();
-    BatchScanner bs = conn.createBatchScanner(opts.tableName, opts.auths, bsOpts.scanThreads);
-    bs.setTimeout(bsOpts.scanTimeout, TimeUnit.MILLISECONDS);
+  public static List<String> query(BatchScanner bs, List<String> terms) {
     
-    Text columns[] = new Text[opts.terms.size()];
+    Text columns[] = new Text[terms.size()];
     int i = 0;
-    for (String term : opts.terms) {
+    for (String term : terms) {
       columns[i++] = new Text(term);
     }
     IteratorSetting ii = new IteratorSetting(20, "ii", IntersectingIterator.class);
     IntersectingIterator.setColumnFamilies(ii, columns);
     bs.addScanIterator(ii);
     bs.setRanges(Collections.singleton(new Range()));
+    List<String> result = new ArrayList<String>();
     for (Entry<Key,Value> entry : bs) {
-      System.out.println("  " + entry.getKey().getColumnQualifier());
+      result.add(entry.getKey().getColumnQualifier().toString());
     }
-    
+    return result;
+  }
+  
+  /**
+   * @param args
+   */
+  public static void main(String[] args) throws Exception {
+    Opts opts = new Opts();
+    BatchScannerOpts bsOpts = new BatchScannerOpts();
+    opts.parseArgs(Query.class.getName(), args, bsOpts);
+    Connector conn = opts.getConnector();
+    BatchScanner bs = conn.createBatchScanner(opts.tableName, opts.auths, bsOpts.scanThreads);
+    bs.setTimeout(bsOpts.scanTimeout, TimeUnit.MILLISECONDS);
+
+    for (String entry : query(bs, opts.terms))
+      System.out.println("  " + entry);
   }
   
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloCluster.java Tue Jul  2 21:25:56 2013
@@ -38,9 +38,11 @@ import org.apache.accumulo.core.client.C
 import org.apache.accumulo.core.client.Instance;
 import org.apache.accumulo.core.client.ZooKeeperInstance;
 import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.master.thrift.MasterGoalState;
 import org.apache.accumulo.core.util.Pair;
 import org.apache.accumulo.core.util.UtilWaitThread;
 import org.apache.accumulo.server.master.Master;
+import org.apache.accumulo.server.master.state.SetGoalState;
 import org.apache.accumulo.server.tabletserver.TabletServer;
 import org.apache.accumulo.server.util.Initialize;
 import org.apache.accumulo.server.util.PortUtils;
@@ -105,6 +107,7 @@ public class MiniAccumuloCluster {
     }
   }
 
+  private boolean initialized = false;
   private Process zooKeeperProcess;
   private Process masterProcess;
   private Process[] tabletServerProcesses;
@@ -122,7 +125,7 @@ public class MiniAccumuloCluster {
 
   private MiniAccumuloConfig config;
 
-  private Process exec(Class<? extends Object> clazz, String... args) throws IOException {
+  public Process exec(Class<? extends Object> clazz, String... args) throws IOException {
     return exec(clazz, Collections.singletonList("-Xmx" + config.getDefaultMemory()), args);
   }
 
@@ -247,34 +250,45 @@ public class MiniAccumuloCluster {
     if (zooKeeperProcess != null)
       throw new IllegalStateException("Already started");
 
-    Runtime.getRuntime().addShutdownHook(new Thread() {
-      @Override
-      public void run() {
-        try {
-          MiniAccumuloCluster.this.stop();
-        } catch (IOException e) {
-          e.printStackTrace();
-        } catch (InterruptedException e) {
-          e.printStackTrace();
+    if (!initialized) {
+      
+      Runtime.getRuntime().addShutdownHook(new Thread() {
+        @Override
+        public void run() {
+          try {
+            MiniAccumuloCluster.this.stop();
+          } catch (IOException e) {
+            e.printStackTrace();
+          } catch (InterruptedException e) {
+            e.printStackTrace();
+          }
         }
-      }
-    });
-
+      });
+    }
+      
     zooKeeperProcess = exec(Main.class, ServerType.ZOOKEEPER, ZooKeeperServerMain.class.getName(), zooCfgFile.getAbsolutePath());
 
     // sleep a little bit to let zookeeper come up before calling init, seems to work better
     UtilWaitThread.sleep(250);
     
-    Process initProcess = exec(Initialize.class, "--instance-name", config.getInstanceName(), "--password", config.getRootPassword(), "--username", "root");
-    int ret = initProcess.waitFor();
-    if (ret != 0) {
-      throw new RuntimeException("Initialize process returned " + ret);
+    if (!initialized) {
+      Process initProcess = exec(Initialize.class, "--instance-name", config.getInstanceName(), "--password", config.getRootPassword(), "--username", "root");
+      int ret = initProcess.waitFor();
+      if (ret != 0) {
+        throw new RuntimeException("Initialize process returned " + ret);
+      }
+      initialized = true; 
     }
 
     tabletServerProcesses = new Process[config.getNumTservers()];
     for (int i = 0; i < config.getNumTservers(); i++) {
       tabletServerProcesses[i] = exec(TabletServer.class, ServerType.TABLET_SERVER);
     }
+    Process goal = exec(Main.class, SetGoalState.class.getName(), MasterGoalState.NORMAL.toString());
+    int ret = goal.waitFor();
+    if (ret != 0) {
+      throw new RuntimeException("Could not set master goal state, process returned " + ret);
+    }
 
     masterProcess = exec(Master.class, ServerType.MASTER);
   }
@@ -326,6 +340,9 @@ public class MiniAccumuloCluster {
 
     for (LogWriter lw : logWriters)
       lw.flush();
+    zooKeeperProcess = null;
+    masterProcess = null;
+    tabletServerProcesses = null;
   }
 
   /**
@@ -335,6 +352,15 @@ public class MiniAccumuloCluster {
     return config;
   }
 
+  /**
+   * Utility method to get a connector to the MAC.
+   * @since 1.6.0
+   * @param user
+   * @param passwd
+   * @return Connector
+   * @throws AccumuloException
+   * @throws AccumuloSecurityException
+   */
   public Connector getConnector(String user, String passwd) throws AccumuloException, AccumuloSecurityException {
     Instance instance = new ZooKeeperInstance(this.getInstanceName(), this.getZooKeepers());
     return instance.getConnector(user, new PasswordToken(passwd));

Modified: accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/minicluster/src/main/java/org/apache/accumulo/minicluster/MiniAccumuloConfig.java Tue Jul  2 21:25:56 2013
@@ -17,10 +17,12 @@
 package org.apache.accumulo.minicluster;
 
 import java.io.File;
+import java.util.Arrays;
 import java.util.HashMap;
 import java.util.Map;
 
 import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.util.StringUtil;
 import org.apache.accumulo.server.util.PortUtils;
 
 /**
@@ -87,9 +89,18 @@ public class MiniAccumuloConfig {
       logDir = new File(dir, "logs");
       walogDir = new File(dir, "walogs");
       
-      String classpath = System.getenv("ACCUMULO_HOME") + "/lib/.*.jar," + "$ZOOKEEPER_HOME/zookeeper[^.].*.jar," + "$HADOOP_HOME/[^.].*.jar,"
-          + "$HADOOP_HOME/lib/[^.].*.jar," + "$HADOOP_PREFIX/share/hadoop/common/.*.jar," + "$HADOOP_PREFIX/share/hadoop/common/lib/.*.jar,"
-          + "$HADOOP_PREFIX/share/hadoop/hdfs/.*.jar," + "$HADOOP_PREFIX/share/hadoop/mapreduce/.*.jar";
+      String[] paths = {
+          "$ACCUMULO_HOME/lib/.*.jar",
+          "$ZOOKEEPER_HOME/zookeeper[^.].*.jar",
+          "$HADOOP_PREFIX/[^.].*.jar",
+          "$HADOOP_PREFIX/lib/[^.].*.jar",
+          "$HADOOP_PREFIX/share/hadoop/common/.*.jar",
+          "$HADOOP_PREFIX/share/hadoop/common/lib/.*.jar",
+          "$HADOOP_PREFIX/share/hadoop/hdfs/.*.jar",
+          "$HADOOP_PREFIX/share/hadoop/mapreduce/.*.jar"
+      };
+      
+      String classpath = StringUtil.join(Arrays.asList(paths), ",");
       
       mergeProp(Property.INSTANCE_DFS_URI.getKey(), "file:///");
       mergeProp(Property.INSTANCE_DFS_DIR.getKey(), accumuloDir.getAbsolutePath());
@@ -109,6 +120,7 @@ public class MiniAccumuloConfig {
       mergePropWithRandomPort(Property.MASTER_CLIENTPORT.getKey());
       mergePropWithRandomPort(Property.TRACE_PORT.getKey());
       mergePropWithRandomPort(Property.TSERV_CLIENTPORT.getKey());
+      mergePropWithRandomPort(Property.MONITOR_PORT.getKey());
       
       // zookeeper port should be set explicitly in this class, not just on the site config
       if (zooKeeperPort == null)

Modified: accumulo/branches/ACCUMULO-CURATOR/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java Tue Jul  2 21:25:56 2013
@@ -1141,6 +1141,6 @@ public class SimpleTest {
   @AfterClass
   public static void tearDownMiniCluster() throws Exception {
     accumulo.stop();
-//    folder.delete();
+    folder.delete();
   }
 }

Propchange: accumulo/branches/ACCUMULO-CURATOR/server/
------------------------------------------------------------------------------
  Merged /accumulo/trunk/server:r1496191-1499112

Modified: accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/master/recovery/HadoopLogCloser.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/master/recovery/HadoopLogCloser.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/master/recovery/HadoopLogCloser.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/master/recovery/HadoopLogCloser.java Tue Jul  2 21:25:56 2013
@@ -43,19 +43,18 @@ public class HadoopLogCloser implements 
           return master.getConfiguration().getConfiguration().getTimeInMillis(Property.MASTER_LEASE_RECOVERY_WAITING_PERIOD);
         }
         log.info("Recovered lease on " + source.toString());
-        return 0;
       } catch (FileNotFoundException ex) {
         throw ex;
       } catch (Exception ex) {
         log.warn("Error recovery lease on " + source.toString(), ex);
+        ns.append(source).close();
+        log.info("Recovered lease on " + source.toString() + " using append");
       }
     } else if (ns instanceof LocalFileSystem) {
       // ignore
     } else {
       throw new IllegalStateException("Don't know how to recover a lease for " + fs.getClass().getName());
     }
-    ns.append(source).close();
-    log.info("Recovered lease on " + source.toString() + " using append");
     return 0;
   }
   

Modified: accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/util/CheckForMetadataProblems.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/util/CheckForMetadataProblems.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/util/CheckForMetadataProblems.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/server/src/main/java/org/apache/accumulo/server/util/CheckForMetadataProblems.java Tue Jul  2 21:25:56 2013
@@ -195,7 +195,7 @@ public class CheckForMetadataProblems {
     checkMetadataTableEntries(opts, fs);
     opts.stopTracing();
     if (sawProblems)
-      System.exit(-1);
+      throw new RuntimeException();
   }
   
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/test/pom.xml
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/test/pom.xml?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/test/pom.xml (original)
+++ accumulo/branches/ACCUMULO-CURATOR/test/pom.xml Tue Jul  2 21:25:56 2013
@@ -106,10 +106,25 @@
       <scope>provided</scope>
     </dependency>
     <dependency>
+      <groupId>commons-httpclient</groupId>
+      <artifactId>commons-httpclient</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
+      <groupId>org.apache.accumulo</groupId>
+      <artifactId>accumulo-examples-simple</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mortbay.jetty</groupId>
+      <artifactId>jetty</artifactId>
+      <scope>test</scope>
+    </dependency>
   </dependencies>
   <build>
     <pluginManagement>

Modified: accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestBinaryRows.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestBinaryRows.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestBinaryRows.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestBinaryRows.java Tue Jul  2 21:25:56 2013
@@ -70,169 +70,159 @@ public class TestBinaryRows {
     return l;
   }
   
-  static class Opts extends ClientOnRequiredTable {
+  public static class Opts extends ClientOnRequiredTable {
     @Parameter(names="--mode", description="either 'ingest', 'delete', 'randomLookups', 'split', 'verify', 'verifyDeleted'", required=true)
-    String mode;
+    public String mode;
     @Parameter(names="--start", description="the lowest numbered row")
-    long start = 0;
+    public long start = 0;
     @Parameter(names="--count", description="number of rows to ingest", required=true)
-    long num = 0;
+    public long num = 0;
   }
   
-  public static void main(String[] args) {
-    Opts opts = new Opts();
-    BatchWriterOpts bwOpts = new BatchWriterOpts();
-    ScannerOpts scanOpts = new ScannerOpts();
-    opts.parseArgs(TestBinaryRows.class.getName(), args, scanOpts, bwOpts);
+  public static void runTest(Connector connector, Opts opts, BatchWriterOpts bwOpts, ScannerOpts scanOpts) throws Exception {
     
-    try {
-      Connector connector = opts.getConnector();
+    if (opts.mode.equals("ingest") || opts.mode.equals("delete")) {
+      BatchWriter bw = connector.createBatchWriter(opts.tableName, bwOpts.getBatchWriterConfig());
+      boolean delete = opts.mode.equals("delete");
       
-      if (opts.mode.equals("ingest") || opts.mode.equals("delete")) {
-        BatchWriter bw = connector.createBatchWriter(opts.tableName, bwOpts.getBatchWriterConfig());
-        boolean delete = opts.mode.equals("delete");
-        
-        for (long i = 0; i < opts.num; i++) {
-          byte[] row = encodeLong(i + opts.start);
-          String value = "" + (i + opts.start);
-          
-          Mutation m = new Mutation(new Text(row));
-          if (delete) {
-            m.putDelete(new Text("cf"), new Text("cq"));
-          } else {
-            m.put(new Text("cf"), new Text("cq"), new Value(value.getBytes()));
-          }
-          bw.addMutation(m);
+      for (long i = 0; i < opts.num; i++) {
+        byte[] row = encodeLong(i + opts.start);
+        String value = "" + (i + opts.start);
+        
+        Mutation m = new Mutation(new Text(row));
+        if (delete) {
+          m.putDelete(new Text("cf"), new Text("cq"));
+        } else {
+          m.put(new Text("cf"), new Text("cq"), new Value(value.getBytes()));
         }
+        bw.addMutation(m);
+      }
+      
+      bw.close();
+    } else if (opts.mode.equals("verifyDeleted")) {
+      Scanner s = connector.createScanner(opts.tableName, opts.auths);
+      s.setBatchSize(scanOpts.scanBatchSize);
+      Key startKey = new Key(encodeLong(opts.start), "cf".getBytes(), "cq".getBytes(), new byte[0], Long.MAX_VALUE);
+      Key stopKey = new Key(encodeLong(opts.start + opts.num - 1), "cf".getBytes(), "cq".getBytes(), new byte[0], 0);
+      s.setBatchSize(50000);
+      s.setRange(new Range(startKey, stopKey));
+      
+      for (Entry<Key,Value> entry : s) {
+        throw new Exception("ERROR : saw entries in range that should be deleted ( first value : " + entry.getValue().toString() + ")");
+      }
+      
+    } else if (opts.mode.equals("verify")) {
+      long t1 = System.currentTimeMillis();
+      
+      Scanner s = connector.createScanner(opts.tableName, opts.auths);
+      Key startKey = new Key(encodeLong(opts.start), "cf".getBytes(), "cq".getBytes(), new byte[0], Long.MAX_VALUE);
+      Key stopKey = new Key(encodeLong(opts.start + opts.num - 1), "cf".getBytes(), "cq".getBytes(), new byte[0], 0);
+      s.setBatchSize(scanOpts.scanBatchSize);
+      s.setRange(new Range(startKey, stopKey));
+      
+      long i = opts.start;
+      
+      for (Entry<Key,Value> e : s) {
+        Key k = e.getKey();
+        Value v = e.getValue();
         
-        bw.close();
-      } else if (opts.mode.equals("verifyDeleted")) {
-        Scanner s = connector.createScanner(opts.tableName, opts.auths);
-        s.setBatchSize(scanOpts.scanBatchSize);
-        Key startKey = new Key(encodeLong(opts.start), "cf".getBytes(), "cq".getBytes(), new byte[0], Long.MAX_VALUE);
-        Key stopKey = new Key(encodeLong(opts.start + opts.num - 1), "cf".getBytes(), "cq".getBytes(), new byte[0], 0);
-        s.setBatchSize(50000);
-        s.setRange(new Range(startKey, stopKey));
+        // System.out.println("v = "+v);
         
-        for (Entry<Key,Value> entry : s) {
-          System.err.println("ERROR : saw entries in range that should be deleted ( first value : " + entry.getValue().toString() + ")");
-          System.err.println("exiting...");
-          System.exit(1);
-        }
+        checkKeyValue(i, k, v);
         
-      } else if (opts.mode.equals("verify")) {
-        long t1 = System.currentTimeMillis();
+        i++;
+      }
+      
+      if (i != opts.start + opts.num) {
+        throw new Exception("ERROR : did not see expected number of rows, saw " + (i - opts.start) + " expected " + opts.num);
+      }
+      
+      long t2 = System.currentTimeMillis();
+      
+      System.out.printf("time : %9.2f secs%n", ((t2 - t1) / 1000.0));
+      System.out.printf("rate : %9.2f entries/sec%n", opts.num / ((t2 - t1) / 1000.0));
+      
+    } else if (opts.mode.equals("randomLookups")) {
+      int numLookups = 1000;
+      
+      Random r = new Random();
+      
+      long t1 = System.currentTimeMillis();
+      
+      for (int i = 0; i < numLookups; i++) {
+        long row = ((r.nextLong() & 0x7fffffffffffffffl) % opts.num) + opts.start;
         
         Scanner s = connector.createScanner(opts.tableName, opts.auths);
-        Key startKey = new Key(encodeLong(opts.start), "cf".getBytes(), "cq".getBytes(), new byte[0], Long.MAX_VALUE);
-        Key stopKey = new Key(encodeLong(opts.start + opts.num - 1), "cf".getBytes(), "cq".getBytes(), new byte[0], 0);
         s.setBatchSize(scanOpts.scanBatchSize);
+        Key startKey = new Key(encodeLong(row), "cf".getBytes(), "cq".getBytes(), new byte[0], Long.MAX_VALUE);
+        Key stopKey = new Key(encodeLong(row), "cf".getBytes(), "cq".getBytes(), new byte[0], 0);
         s.setRange(new Range(startKey, stopKey));
         
-        long i = opts.start;
+        Iterator<Entry<Key,Value>> si = s.iterator();
         
-        for (Entry<Key,Value> e : s) {
+        if (si.hasNext()) {
+          Entry<Key,Value> e = si.next();
           Key k = e.getKey();
           Value v = e.getValue();
           
-          // System.out.println("v = "+v);
-          
-          checkKeyValue(i, k, v);
-          
-          i++;
-        }
-        
-        if (i != opts.start + opts.num) {
-          System.err.println("ERROR : did not see expected number of rows, saw " + (i - opts.start) + " expected " + opts.num);
-          System.err.println("exiting... ARGHHHHHH");
-          System.exit(1);
-          
-        }
-        
-        long t2 = System.currentTimeMillis();
-        
-        System.out.printf("time : %9.2f secs%n", ((t2 - t1) / 1000.0));
-        System.out.printf("rate : %9.2f entries/sec%n", opts.num / ((t2 - t1) / 1000.0));
-        
-      } else if (opts.mode.equals("randomLookups")) {
-        int numLookups = 1000;
-        
-        Random r = new Random();
-        
-        long t1 = System.currentTimeMillis();
-        
-        for (int i = 0; i < numLookups; i++) {
-          long row = ((r.nextLong() & 0x7fffffffffffffffl) % opts.num) + opts.start;
-          
-          Scanner s = connector.createScanner(opts.tableName, opts.auths);
-          s.setBatchSize(scanOpts.scanBatchSize);
-          Key startKey = new Key(encodeLong(row), "cf".getBytes(), "cq".getBytes(), new byte[0], Long.MAX_VALUE);
-          Key stopKey = new Key(encodeLong(row), "cf".getBytes(), "cq".getBytes(), new byte[0], 0);
-          s.setRange(new Range(startKey, stopKey));
-          
-          Iterator<Entry<Key,Value>> si = s.iterator();
+          checkKeyValue(row, k, v);
           
           if (si.hasNext()) {
-            Entry<Key,Value> e = si.next();
-            Key k = e.getKey();
-            Value v = e.getValue();
-            
-            checkKeyValue(row, k, v);
-            
-            if (si.hasNext()) {
-              System.err.println("ERROR : lookup on " + row + " returned more than one result ");
-              System.err.println("exiting...");
-              System.exit(1);
-            }
-            
-          } else {
-            System.err.println("ERROR : lookup on " + row + " failed ");
-            System.err.println("exiting...");
-            System.exit(1);
+            throw new Exception("ERROR : lookup on " + row + " returned more than one result ");
           }
-        }
-        
-        long t2 = System.currentTimeMillis();
-        
-        System.out.printf("time    : %9.2f secs%n", ((t2 - t1) / 1000.0));
-        System.out.printf("lookups : %9d keys%n", numLookups);
-        System.out.printf("rate    : %9.2f lookups/sec%n", numLookups / ((t2 - t1) / 1000.0));
-        
-      } else if (opts.mode.equals("split")) {
-        TreeSet<Text> splits = new TreeSet<Text>();
-        int shift = (int) opts.start;
-        int count = (int) opts.num;
-        
-        for (long i = 0; i < count; i++) {
-          long splitPoint = i << shift;
           
-          splits.add(new Text(encodeLong(splitPoint)));
-          System.out.printf("added split point 0x%016x  %,12d%n", splitPoint, splitPoint);
+        } else {
+          throw new Exception("ERROR : lookup on " + row + " failed ");
         }
+      }
+      
+      long t2 = System.currentTimeMillis();
+      
+      System.out.printf("time    : %9.2f secs%n", ((t2 - t1) / 1000.0));
+      System.out.printf("lookups : %9d keys%n", numLookups);
+      System.out.printf("rate    : %9.2f lookups/sec%n", numLookups / ((t2 - t1) / 1000.0));
+      
+    } else if (opts.mode.equals("split")) {
+      TreeSet<Text> splits = new TreeSet<Text>();
+      int shift = (int) opts.start;
+      int count = (int) opts.num;
+      
+      for (long i = 0; i < count; i++) {
+        long splitPoint = i << shift;
         
-        connector.tableOperations().create(opts.tableName);
-        connector.tableOperations().addSplits(opts.tableName, splits);
-        
-      } else {
-        System.err.println("ERROR : " + opts.mode + " is not a valid operation.");
-        System.exit(1);
+        splits.add(new Text(encodeLong(splitPoint)));
+        System.out.printf("added split point 0x%016x  %,12d%n", splitPoint, splitPoint);
       }
-    } catch (Exception e) {
-      throw new RuntimeException(e);
+      
+      connector.tableOperations().create(opts.tableName);
+      connector.tableOperations().addSplits(opts.tableName, splits);
+      
+    } else {
+      throw new Exception("ERROR : " + opts.mode + " is not a valid operation.");
     }
   }
   
   private static void checkKeyValue(long expected, Key k, Value v) throws Exception {
     if (expected != decodeLong(TextUtil.getBytes(k.getRow()))) {
-      System.err.println("ERROR : expected row " + expected + " saw " + decodeLong(TextUtil.getBytes(k.getRow())));
-      System.err.println("exiting...");
-      throw new Exception();
+      throw new Exception("ERROR : expected row " + expected + " saw " + decodeLong(TextUtil.getBytes(k.getRow())));
     }
     
     if (!v.toString().equals("" + expected)) {
-      System.err.println("ERROR : expected value " + expected + " saw " + v.toString());
-      System.err.println("exiting...");
-      throw new Exception();
+      throw new Exception("ERROR : expected value " + expected + " saw " + v.toString());
+    }
+  }
+
+  public static void main(String[] args) {
+    Opts opts = new Opts();
+    BatchWriterOpts bwOpts = new BatchWriterOpts();
+    ScannerOpts scanOpts = new ScannerOpts();
+    opts.parseArgs(TestBinaryRows.class.getName(), args, scanOpts, bwOpts);
+    
+    try {
+      runTest(opts.getConnector(), opts, bwOpts, scanOpts);
+    } catch (Exception e) {
+      throw new RuntimeException(e);
     }
   }
+
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestIngest.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestIngest.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestIngest.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestIngest.java Tue Jul  2 21:25:56 2013
@@ -16,16 +16,20 @@
  */
 package org.apache.accumulo.test;
 
+import java.io.IOException;
 import java.util.Map.Entry;
 import java.util.Random;
 import java.util.Set;
 import java.util.TreeSet;
 
 import org.apache.accumulo.core.cli.BatchWriterOpts;
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
 import org.apache.accumulo.core.client.BatchWriter;
 import org.apache.accumulo.core.client.Connector;
 import org.apache.accumulo.core.client.Instance;
 import org.apache.accumulo.core.client.MutationsRejectedException;
+import org.apache.accumulo.core.client.TableExistsException;
 import org.apache.accumulo.core.client.TableNotFoundException;
 import org.apache.accumulo.core.client.impl.TabletServerBatchWriter;
 import org.apache.accumulo.core.client.security.SecurityErrorCode;
@@ -59,58 +63,57 @@ import com.beust.jcommander.Parameter;
 public class TestIngest {
   public static final Authorizations AUTHS = new Authorizations("L1", "L2", "G1", "GROUP2");
   
-  static class Opts extends ClientOnDefaultTable {
+  public static class Opts extends ClientOnDefaultTable {
     
     @Parameter(names="--createTable")
-    boolean createTable = false;
+    public boolean createTable = false;
     
     @Parameter(names="--splits", description="the number of splits to use when creating the table")
-    int numsplits = 1;
+    public int numsplits = 1;
     
     @Parameter(names="--start", description="the starting row number")
-    int startRow = 0;
+    public int startRow = 0;
     
     @Parameter(names="--rows", description="the number of rows to ingest")
-    int rows = 100000;
+    public int rows = 100000;
     
     @Parameter(names="--cols", description="the number of columns to ingest per row")
-    int cols = 1;
+    public int cols = 1;
     
     @Parameter(names="--random", description="insert random rows and use the given number to seed the psuedo-random number generator")
-    Integer random = null;
+    public Integer random = null;
     
     @Parameter(names="--size", description="the size of the value to ingest")
-    int dataSize = 1000;
+    public int dataSize = 1000;
     
     @Parameter(names="--delete", description="delete values instead of inserting them")
-    boolean delete = false;
+    public boolean delete = false;
     
     @Parameter(names={"-ts", "--timestamp"}, description="timestamp to use for all values")
-    long timestamp = -1;
+    public long timestamp = -1;
     
     @Parameter(names="--rfile", description="generate data into a file that can be imported")
-    String outputFile = null;
+    public String outputFile = null;
     
     @Parameter(names="--stride", description="the difference between successive row ids")
-    int stride;
+    public int stride;
 
     @Parameter(names={"-cf","--columnFamily"}, description="place columns in this column family")
-    String columnFamily = "colf";
+    public String columnFamily = "colf";
 
     @Parameter(names={"-cv","--columnVisibility"}, description="place columns in this column family", converter=VisibilityConverter.class)
-    ColumnVisibility columnVisibility = new ColumnVisibility();
+    public ColumnVisibility columnVisibility = new ColumnVisibility();
 
-    Opts() { super("test_ingest"); }
+    public Opts() { super("test_ingest"); }
   }
   
   @SuppressWarnings("unused")
   private static final Logger log = Logger.getLogger(TestIngest.class);
   
-  public static void createTable(Opts args) throws Exception {
+  public static void createTable(Connector conn, Opts args) throws AccumuloException, AccumuloSecurityException, TableExistsException {
     if (args.createTable) {
       TreeSet<Text> splits = getSplitPoints(args.startRow, args.startRow + args.rows, args.numsplits);
       
-      Connector conn = args.getConnector();
       if (!conn.tableOperations().exists(args.getTableName()))
         conn.tableOperations().create(args.getTableName());
       try {
@@ -136,15 +139,15 @@ public class TestIngest {
     return splits;
   }
   
-  public static byte[][] generateValues(Opts ingestArgs) {
+  public static byte[][] generateValues(int dataSize) {
     
     byte[][] bytevals = new byte[10][];
     
     byte[] letters = {'1', '2', '3', '4', '5', '6', '7', '8', '9', '0'};
     
     for (int i = 0; i < 10; i++) {
-      bytevals[i] = new byte[ingestArgs.dataSize];
-      for (int j = 0; j < ingestArgs.dataSize; j++)
+      bytevals[i] = new byte[dataSize];
+      for (int j = 0; j < dataSize; j++)
         bytevals[i][j] = letters[i];
     }
     return bytevals;
@@ -179,7 +182,6 @@ public class TestIngest {
     opts.parseArgs(TestIngest.class.getName(), args, bwOpts);
     opts.getInstance().setConfiguration(ServerConfiguration.getSiteConfiguration());
 
-    createTable(opts);
     
     Instance instance = opts.getInstance();
     
@@ -193,143 +195,149 @@ public class TestIngest {
       
       // test batch update
       
-      long stopTime;
-      
-      byte[][] bytevals = generateValues(opts);
-      
-      byte randomValue[] = new byte[opts.dataSize];
-      Random random = new Random();
-      
-      long bytesWritten = 0;
-      
-      BatchWriter bw = null;
-      FileSKVWriter writer = null;
-      
-      if (opts.outputFile != null) {
-        Configuration conf = CachedConfiguration.getInstance();
-        FileSystem fs = FileSystem.get(conf);
-        writer = FileOperations.getInstance().openWriter(opts.outputFile + "." + RFile.EXTENSION, fs, conf,
-            AccumuloConfiguration.getDefaultConfiguration());
-        writer.startDefaultLocalityGroup();
+      ingest(opts.getConnector(), opts, bwOpts);
+    } catch (Exception e) {
+      throw new RuntimeException(e);
+    } finally {
+      Trace.off();
+    }
+  }
+
+  public static void ingest(Connector connector, Opts opts, BatchWriterOpts bwOpts) throws IOException, AccumuloException, AccumuloSecurityException, TableNotFoundException,
+      MutationsRejectedException, TableExistsException {
+    long stopTime;
+    
+    byte[][] bytevals = generateValues(opts.dataSize);
+    
+    byte randomValue[] = new byte[opts.dataSize];
+    Random random = new Random();
+    
+    long bytesWritten = 0;
+
+    createTable(connector, opts);
+    
+    BatchWriter bw = null;
+    FileSKVWriter writer = null;
+    
+    if (opts.outputFile != null) {
+      Configuration conf = CachedConfiguration.getInstance();
+      FileSystem fs = FileSystem.get(conf);
+      writer = FileOperations.getInstance().openWriter(opts.outputFile + "." + RFile.EXTENSION, fs, conf,
+          AccumuloConfiguration.getDefaultConfiguration());
+      writer.startDefaultLocalityGroup();
+    } else {
+      bw = connector.createBatchWriter(opts.getTableName(), bwOpts.getBatchWriterConfig());
+      connector.securityOperations().changeUserAuthorizations(opts.principal, AUTHS);
+    }
+    Text labBA = new Text(opts.columnVisibility.getExpression());
+    
+    long startTime = System.currentTimeMillis();
+    for (int i = 0; i < opts.rows; i++) {
+      int rowid;
+      if (opts.stride > 0) {
+        rowid = ((i % opts.stride) * (opts.rows / opts.stride)) + (i / opts.stride);
       } else {
-        Connector connector = opts.getConnector();
-        bw = connector.createBatchWriter(opts.getTableName(), bwOpts.getBatchWriterConfig());
-        connector.securityOperations().changeUserAuthorizations(opts.principal, AUTHS);
+        rowid = i;
       }
-      Text labBA = new Text(opts.columnVisibility.getExpression());
       
-      long startTime = System.currentTimeMillis();
-      for (int i = 0; i < opts.rows; i++) {
-        int rowid;
-        if (opts.stride > 0) {
-          rowid = ((i % opts.stride) * (opts.rows / opts.stride)) + (i / opts.stride);
-        } else {
-          rowid = i;
-        }
+      Text row = generateRow(rowid, opts.startRow);
+      Mutation m = new Mutation(row);
+      for (int j = 0; j < opts.cols; j++) {
+        Text colf = new Text(opts.columnFamily);
+        Text colq = new Text(FastFormat.toZeroPaddedString(j, 7, 10, COL_PREFIX));
         
-        Text row = generateRow(rowid, opts.startRow);
-        Mutation m = new Mutation(row);
-        for (int j = 0; j < opts.cols; j++) {
-          Text colf = new Text(opts.columnFamily);
-          Text colq = new Text(FastFormat.toZeroPaddedString(j, 7, 10, COL_PREFIX));
+        if (writer != null) {
+          Key key = new Key(row, colf, colq, labBA);
+          if (opts.timestamp >= 0) {
+            key.setTimestamp(opts.timestamp);
+          } else {
+            key.setTimestamp(startTime);
+          }
           
-          if (writer != null) {
-            Key key = new Key(row, colf, colq, labBA);
-            if (opts.timestamp >= 0) {
-              key.setTimestamp(opts.timestamp);
-            } else {
-              key.setTimestamp(startTime);
-            }
-            
-            if (opts.delete) {
-              key.setDeleted(true);
+          if (opts.delete) {
+            key.setDeleted(true);
+          } else {
+            key.setDeleted(false);
+          }
+          
+          bytesWritten += key.getSize();
+          
+          if (opts.delete) {
+            writer.append(key, new Value(new byte[0]));
+          } else {
+            byte value[];
+            if (opts.random != null) {
+              value = genRandomValue(random, randomValue, opts.random.intValue(), rowid + opts.startRow, j);
             } else {
-              key.setDeleted(false);
+              value = bytevals[j % bytevals.length];
             }
             
-            bytesWritten += key.getSize();
-            
-            if (opts.delete) {
-              writer.append(key, new Value(new byte[0]));
+            Value v = new Value(value);
+            writer.append(key, v);
+            bytesWritten += v.getSize();
+          }
+          
+        } else {
+          Key key = new Key(row, colf, colq, labBA);
+          bytesWritten += key.getSize();
+          
+          if (opts.delete) {
+            if (opts.timestamp >= 0)
+              m.putDelete(colf, colq, opts.columnVisibility, opts.timestamp);
+            else
+              m.putDelete(colf, colq, opts.columnVisibility);
+          } else {
+            byte value[];
+            if (opts.random != null) {
+              value = genRandomValue(random, randomValue, opts.random.intValue(), rowid + opts.startRow, j);
             } else {
-              byte value[];
-              if (opts.random != null) {
-                value = genRandomValue(random, randomValue, opts.random.intValue(), rowid + opts.startRow, j);
-              } else {
-                value = bytevals[j % bytevals.length];
-              }
-              
-              Value v = new Value(value);
-              writer.append(key, v);
-              bytesWritten += v.getSize();
+              value = bytevals[j % bytevals.length];
             }
+            bytesWritten += value.length;
             
-          } else {
-            Key key = new Key(row, colf, colq, labBA);
-            bytesWritten += key.getSize();
-            
-            if (opts.delete) {
-              if (opts.timestamp >= 0)
-                m.putDelete(colf, colq, opts.columnVisibility, opts.timestamp);
-              else
-                m.putDelete(colf, colq, opts.columnVisibility);
+            if (opts.timestamp >= 0) {
+              m.put(colf, colq, opts.columnVisibility, opts.timestamp, new Value(value, true));
             } else {
-              byte value[];
-              if (opts.random != null) {
-                value = genRandomValue(random, randomValue, opts.random.intValue(), rowid + opts.startRow, j);
-              } else {
-                value = bytevals[j % bytevals.length];
-              }
-              bytesWritten += value.length;
+              m.put(colf, colq, opts.columnVisibility, new Value(value, true));
               
-              if (opts.timestamp >= 0) {
-                m.put(colf, colq, opts.columnVisibility, opts.timestamp, new Value(value, true));
-              } else {
-                m.put(colf, colq, opts.columnVisibility, new Value(value, true));
-                
-              }
             }
           }
-          
         }
-        if (bw != null)
-          bw.addMutation(m);
         
       }
+      if (bw != null)
+        bw.addMutation(m);
       
-      if (writer != null) {
-        writer.close();
-      } else if (bw != null) {
-        try {
-          bw.close();
-        } catch (MutationsRejectedException e) {
-          if (e.getAuthorizationFailuresMap().size() > 0) {
-            for (Entry<KeyExtent,Set<SecurityErrorCode>> entry : e.getAuthorizationFailuresMap().entrySet()) {
-              System.err.println("ERROR : Not authorized to write to : " + entry.getKey() + " due to " + entry.getValue());
-            }
+    }
+    
+    if (writer != null) {
+      writer.close();
+    } else if (bw != null) {
+      try {
+        bw.close();
+      } catch (MutationsRejectedException e) {
+        if (e.getAuthorizationFailuresMap().size() > 0) {
+          for (Entry<KeyExtent,Set<SecurityErrorCode>> entry : e.getAuthorizationFailuresMap().entrySet()) {
+            System.err.println("ERROR : Not authorized to write to : " + entry.getKey() + " due to " + entry.getValue());
           }
-          
-          if (e.getConstraintViolationSummaries().size() > 0) {
-            for (ConstraintViolationSummary cvs : e.getConstraintViolationSummaries()) {
-              System.err.println("ERROR : Constraint violates : " + cvs);
-            }
+        }
+        
+        if (e.getConstraintViolationSummaries().size() > 0) {
+          for (ConstraintViolationSummary cvs : e.getConstraintViolationSummaries()) {
+            System.err.println("ERROR : Constraint violates : " + cvs);
           }
-          
-          throw e;
         }
+        
+        throw e;
       }
-      
-      stopTime = System.currentTimeMillis();
-      
-      int totalValues = opts.rows * opts.cols;
-      double elapsed = (stopTime - startTime) / 1000.0;
-      
-      System.out.printf("%,12d records written | %,8d records/sec | %,12d bytes written | %,8d bytes/sec | %6.3f secs   %n", totalValues,
-          (int) (totalValues / elapsed), bytesWritten, (int) (bytesWritten / elapsed), elapsed);
-    } catch (Exception e) {
-      throw new RuntimeException(e);
-    } finally {
-      Trace.off();
     }
+    
+    stopTime = System.currentTimeMillis();
+    
+    int totalValues = opts.rows * opts.cols;
+    double elapsed = (stopTime - startTime) / 1000.0;
+    
+    System.out.printf("%,12d records written | %,8d records/sec | %,12d bytes written | %,8d bytes/sec | %6.3f secs   %n", totalValues,
+        (int) (totalValues / elapsed), bytesWritten, (int) (bytesWritten / elapsed), elapsed);
   }
 }

Modified: accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/TestMultiTableIngest.java Tue Jul  2 21:25:56 2013
@@ -17,6 +17,7 @@
 package org.apache.accumulo.test;
 
 import java.util.ArrayList;
+import java.util.List;
 import java.util.Map.Entry;
 
 import org.apache.accumulo.core.cli.BatchWriterOpts;
@@ -37,8 +38,6 @@ import com.beust.jcommander.Parameter;
 
 public class TestMultiTableIngest {
   
-  private static ArrayList<String> tableNames = new ArrayList<String>();
-  
   static class Opts extends ClientOpts {
     @Parameter(names="--readonly", description="read only")
     boolean readonly = false;
@@ -48,7 +47,7 @@ public class TestMultiTableIngest {
     int count = 10000;
   }
   
-  private static void readBack(Opts opts, ScannerOpts scanOpts, Connector conn) throws Exception {
+  private static void readBack(Opts opts, ScannerOpts scanOpts, Connector conn, List<String> tableNames) throws Exception {
     int i = 0;
     for (String table : tableNames) {
       Scanner scanner = conn.createScanner(table, opts.auths);
@@ -65,6 +64,8 @@ public class TestMultiTableIngest {
   }
   
   public static void main(String[] args) throws Exception {
+    ArrayList<String> tableNames = new ArrayList<String>();
+    
     Opts opts = new Opts();
     ScannerOpts scanOpts = new ScannerOpts();
     BatchWriterOpts bwOpts = new BatchWriterOpts();
@@ -106,7 +107,7 @@ public class TestMultiTableIngest {
       }
     }
     try {
-      readBack(opts, scanOpts, connector);
+      readBack(opts, scanOpts, connector, tableNames);
     } catch (Exception e) {
       throw new RuntimeException(e);
     }

Modified: accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/VerifyIngest.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/VerifyIngest.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/VerifyIngest.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/VerifyIngest.java Tue Jul  2 21:25:56 2013
@@ -22,9 +22,12 @@ import java.util.Map.Entry;
 import java.util.Random;
 
 import org.apache.accumulo.core.cli.ScannerOpts;
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
 import org.apache.accumulo.core.client.Connector;
 import org.apache.accumulo.core.client.Instance;
 import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
 import org.apache.accumulo.core.data.Key;
 import org.apache.accumulo.core.data.PartialKey;
 import org.apache.accumulo.core.data.Range;
@@ -52,10 +55,10 @@ public class VerifyIngest {
   
   public static class Opts extends TestIngest.Opts {
     @Parameter(names = "-useGet", description = "fetches values one at a time, instead of scanning")
-    boolean useGet = false;
+    public boolean useGet = false;
   }
   
-  public static void main(String[] args) {
+  public static void main(String[] args) throws Exception {
     Opts opts = new Opts();
     ScannerOpts scanOpts = new ScannerOpts();
     opts.parseArgs(VerifyIngest.class.getName(), args, scanOpts);
@@ -68,66 +71,138 @@ public class VerifyIngest {
         Trace.currentTrace().data("cmdLine", Arrays.asList(args).toString());
       }
       
-      Connector connector = opts.getConnector();
+      verifyIngest(opts.getConnector(), opts, scanOpts);
       
-      byte[][] bytevals = TestIngest.generateValues(opts);
-      
-      Authorizations labelAuths = new Authorizations("L1", "L2", "G1", "GROUP2");
-      connector.securityOperations().changeUserAuthorizations(opts.principal, labelAuths);
-      
-      int expectedRow = opts.startRow;
-      int expectedCol = 0;
-      int recsRead = 0;
-      
-      long bytesRead = 0;
-      long t1 = System.currentTimeMillis();
-      
-      byte randomValue[] = new byte[opts.dataSize];
-      Random random = new Random();
-      
-      Key endKey = new Key(new Text("row_" + String.format("%010d", opts.rows + opts.startRow)));
-      
-      int errors = 0;
-      
-      while (expectedRow < (opts.rows + opts.startRow)) {
+    } finally {
+      Trace.off();
+    }
+  }
+
+  public static void verifyIngest(Connector connector, Opts opts, ScannerOpts scanOpts) throws AccumuloException, AccumuloSecurityException,
+      TableNotFoundException {
+    byte[][] bytevals = TestIngest.generateValues(opts.dataSize);
+    
+    Authorizations labelAuths = new Authorizations("L1", "L2", "G1", "GROUP2");
+    connector.securityOperations().changeUserAuthorizations(opts.principal, labelAuths);
+    
+    int expectedRow = opts.startRow;
+    int expectedCol = 0;
+    int recsRead = 0;
+    
+    long bytesRead = 0;
+    long t1 = System.currentTimeMillis();
+    
+    byte randomValue[] = new byte[opts.dataSize];
+    Random random = new Random();
+    
+    Key endKey = new Key(new Text("row_" + String.format("%010d", opts.rows + opts.startRow)));
+    
+    int errors = 0;
+    
+    while (expectedRow < (opts.rows + opts.startRow)) {
+      
+      if (opts.useGet) {
+        Text rowKey = new Text("row_" + String.format("%010d", expectedRow + opts.startRow));
+        Text colf = new Text(opts.columnFamily);
+        Text colq = new Text("col_" + String.format("%07d", expectedCol));
+        
+        Scanner scanner = connector.createScanner("test_ingest", labelAuths);
+        scanner.setBatchSize(1);
+        Key startKey = new Key(rowKey, colf, colq);
+        Range range = new Range(startKey, startKey.followingKey(PartialKey.ROW_COLFAM_COLQUAL));
+        scanner.setRange(range);
+        
+        byte[] val = null; // t.get(rowKey, column);
+        
+        Iterator<Entry<Key,Value>> iter = scanner.iterator();
+        
+        if (iter.hasNext()) {
+          val = iter.next().getValue().get();
+        }
         
-        if (opts.useGet) {
-          Text rowKey = new Text("row_" + String.format("%010d", expectedRow + opts.startRow));
-          Text colf = new Text(opts.columnFamily);
-          Text colq = new Text("col_" + String.format("%07d", expectedCol));
+        byte ev[];
+        if (opts.random != null) {
+          ev = TestIngest.genRandomValue(random, randomValue, opts.random.intValue(), expectedRow, expectedCol);
+        } else {
+          ev = bytevals[expectedCol % bytevals.length];
+        }
+        
+        if (val == null) {
+          log.error("Did not find " + rowKey + " " + colf + " " + colq);
+          errors++;
+        } else {
+          recsRead++;
+          bytesRead += val.length;
+          Value value = new Value(val);
+          if (value.compareTo(ev) != 0) {
+            log.error("unexpected value  (" + rowKey + " " + colf + " " + colq + " : saw " + value + " expected " + new Value(ev));
+            errors++;
+          }
+        }
+        
+        expectedCol++;
+        if (expectedCol >= opts.cols) {
+          expectedCol = 0;
+          expectedRow++;
+        }
+        
+      } else {
+        
+        Key startKey = new Key(new Text("row_" + String.format("%010d", expectedRow)));
+        
+        Scanner scanner = connector.createScanner("test_ingest", labelAuths);
+        scanner.setBatchSize(scanOpts.scanBatchSize);
+        scanner.setRange(new Range(startKey, endKey));
+        for (int j = 0; j < opts.cols; j++) {
+          scanner.fetchColumn(new Text(opts.columnFamily), new Text("col_" + String.format("%07d", j)));
+        }
+        
+        int recsReadBefore = recsRead;
+        
+        for (Entry<Key,Value> entry : scanner) {
           
-          Scanner scanner = connector.createScanner("test_ingest", labelAuths);
-          scanner.setBatchSize(1);
-          Key startKey = new Key(rowKey, colf, colq);
-          Range range = new Range(startKey, startKey.followingKey(PartialKey.ROW_COLFAM_COLQUAL));
-          scanner.setRange(range);
+          recsRead++;
           
-          byte[] val = null; // t.get(rowKey, column);
+          bytesRead += entry.getKey().getLength();
+          bytesRead += entry.getValue().getSize();
           
-          Iterator<Entry<Key,Value>> iter = scanner.iterator();
+          int rowNum = getRow(entry.getKey());
+          int colNum = getCol(entry.getKey());
           
-          if (iter.hasNext()) {
-            val = iter.next().getValue().get();
+          if (rowNum != expectedRow) {
+            log.error("rowNum != expectedRow   " + rowNum + " != " + expectedRow);
+            errors++;
+            expectedRow = rowNum;
           }
           
-          byte ev[];
+          if (colNum != expectedCol) {
+            log.error("colNum != expectedCol  " + colNum + " != " + expectedCol + "  rowNum : " + rowNum);
+            errors++;
+          }
+          
+          if (expectedRow >= (opts.rows + opts.startRow)) {
+            log.error("expectedRow (" + expectedRow + ") >= (ingestArgs.rows + ingestArgs.startRow)  (" + (opts.rows + opts.startRow)
+                + "), get batch returned data passed end key");
+            errors++;
+            break;
+          }
+          
+          byte value[];
           if (opts.random != null) {
-            ev = TestIngest.genRandomValue(random, randomValue, opts.random.intValue(), expectedRow, expectedCol);
+            value = TestIngest.genRandomValue(random, randomValue, opts.random.intValue(), expectedRow, colNum);
           } else {
-            ev = bytevals[expectedCol % bytevals.length];
+            value = bytevals[colNum % bytevals.length];
           }
           
-          if (val == null) {
-            log.error("Did not find " + rowKey + " " + colf + " " + colq);
+          if (entry.getValue().compareTo(value) != 0) {
+            log.error("unexpected value, rowNum : " + rowNum + " colNum : " + colNum);
+            log.error(" saw = " + new String(entry.getValue().get()) + " expected = " + new String(value));
+            errors++;
+          }
+          
+          if (opts.timestamp >= 0 && entry.getKey().getTimestamp() != opts.timestamp) {
+            log.error("unexpected timestamp " + entry.getKey().getTimestamp() + ", rowNum : " + rowNum + " colNum : " + colNum);
             errors++;
-          } else {
-            recsRead++;
-            bytesRead += val.length;
-            Value value = new Value(val);
-            if (value.compareTo(ev) != 0) {
-              log.error("unexpected value  (" + rowKey + " " + colf + " " + colq + " : saw " + value + " expected " + new Value(ev));
-              errors++;
-            }
           }
           
           expectedCol++;
@@ -136,100 +211,27 @@ public class VerifyIngest {
             expectedRow++;
           }
           
-        } else {
-          
-          Key startKey = new Key(new Text("row_" + String.format("%010d", expectedRow)));
-          
-          Scanner scanner = connector.createScanner("test_ingest", labelAuths);
-          scanner.setBatchSize(scanOpts.scanBatchSize);
-          scanner.setRange(new Range(startKey, endKey));
-          for (int j = 0; j < opts.cols; j++) {
-            scanner.fetchColumn(new Text(opts.columnFamily), new Text("col_" + String.format("%07d", j)));
-          }
-          
-          int recsReadBefore = recsRead;
-          
-          for (Entry<Key,Value> entry : scanner) {
-            
-            recsRead++;
-            
-            bytesRead += entry.getKey().getLength();
-            bytesRead += entry.getValue().getSize();
-            
-            int rowNum = getRow(entry.getKey());
-            int colNum = getCol(entry.getKey());
-            
-            if (rowNum != expectedRow) {
-              log.error("rowNum != expectedRow   " + rowNum + " != " + expectedRow);
-              errors++;
-              expectedRow = rowNum;
-            }
-            
-            if (colNum != expectedCol) {
-              log.error("colNum != expectedCol  " + colNum + " != " + expectedCol + "  rowNum : " + rowNum);
-              errors++;
-            }
-            
-            if (expectedRow >= (opts.rows + opts.startRow)) {
-              log.error("expectedRow (" + expectedRow + ") >= (ingestArgs.rows + ingestArgs.startRow)  (" + (opts.rows + opts.startRow)
-                  + "), get batch returned data passed end key");
-              errors++;
-              break;
-            }
-            
-            byte value[];
-            if (opts.random != null) {
-              value = TestIngest.genRandomValue(random, randomValue, opts.random.intValue(), expectedRow, colNum);
-            } else {
-              value = bytevals[colNum % bytevals.length];
-            }
-            
-            if (entry.getValue().compareTo(value) != 0) {
-              log.error("unexpected value, rowNum : " + rowNum + " colNum : " + colNum);
-              log.error(" saw = " + new String(entry.getValue().get()) + " expected = " + new String(value));
-              errors++;
-            }
-            
-            if (opts.timestamp >= 0 && entry.getKey().getTimestamp() != opts.timestamp) {
-              log.error("unexpected timestamp " + entry.getKey().getTimestamp() + ", rowNum : " + rowNum + " colNum : " + colNum);
-              errors++;
-            }
-            
-            expectedCol++;
-            if (expectedCol >= opts.cols) {
-              expectedCol = 0;
-              expectedRow++;
-            }
-            
-          }
-          
-          if (recsRead == recsReadBefore) {
-            log.warn("Scan returned nothing, breaking...");
-            break;
-          }
-          
         }
+        
+        if (recsRead == recsReadBefore) {
+          log.warn("Scan returned nothing, breaking...");
+          break;
+        }
+        
       }
-      
-      long t2 = System.currentTimeMillis();
-      
-      if (errors > 0) {
-        log.error("saw " + errors + " errors ");
-        System.exit(1);
-      }
-      
-      if (expectedRow != (opts.rows + opts.startRow)) {
-        log.error("Did not read expected number of rows. Saw " + (expectedRow - opts.startRow) + " expected " + opts.rows);
-        System.exit(1);
-      } else {
-        System.out.printf("%,12d records read | %,8d records/sec | %,12d bytes read | %,8d bytes/sec | %6.3f secs   %n", recsRead,
-            (int) ((recsRead) / ((t2 - t1) / 1000.0)), bytesRead, (int) (bytesRead / ((t2 - t1) / 1000.0)), (t2 - t1) / 1000.0);
-      }
-      
-    } catch (Exception e) {
-      throw new RuntimeException(e);
-    } finally {
-      Trace.off();
+    }
+    
+    long t2 = System.currentTimeMillis();
+    
+    if (errors > 0) {
+      throw new AccumuloException("saw " + errors + " errors ");
+    }
+    
+    if (expectedRow != (opts.rows + opts.startRow)) {
+      throw new AccumuloException("Did not read expected number of rows. Saw " + (expectedRow - opts.startRow) + " expected " + opts.rows);
+    } else {
+      System.out.printf("%,12d records read | %,8d records/sec | %,12d bytes read | %,8d bytes/sec | %6.3f secs   %n", recsRead,
+          (int) ((recsRead) / ((t2 - t1) / 1000.0)), bytesRead, (int) (bytesRead / ((t2 - t1) / 1000.0)), (t2 - t1) / 1000.0);
     }
   }
   

Modified: accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java
URL: http://svn.apache.org/viewvc/accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java?rev=1499116&r1=1499115&r2=1499116&view=diff
==============================================================================
--- accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java (original)
+++ accumulo/branches/ACCUMULO-CURATOR/test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java Tue Jul  2 21:25:56 2013
@@ -107,6 +107,7 @@ public class ContinuousBatchWalker {
       
       count++;
     }
+    bs.close();
     
     long t2 = System.currentTimeMillis();
     



Mime
View raw message