accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-626) create an iterator fuzz tester
Date Mon, 09 Nov 2015 18:37:11 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997078#comment-14997078
] 

ASF GitHub Bot commented on ACCUMULO-626:
-----------------------------------------

Github user keith-turner commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/50#discussion_r44311672
  
    --- Diff: iterator-test-harness/src/test/java/org/apache/accumulo/iteratortest/WholeRowIteratorTest.java
---
    @@ -0,0 +1,147 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to you under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.accumulo.iteratortest;
    +
    +import java.io.IOException;
    +import java.util.ArrayList;
    +import java.util.Collections;
    +import java.util.List;
    +import java.util.Map.Entry;
    +import java.util.TreeMap;
    +
    +import org.apache.accumulo.core.data.Key;
    +import org.apache.accumulo.core.data.Range;
    +import org.apache.accumulo.core.data.Value;
    +import org.apache.accumulo.core.iterators.user.WholeRowIterator;
    +import org.apache.accumulo.iteratortest.junit4.BaseJUnit4IteratorTest;
    +import org.apache.accumulo.iteratortest.testcases.IteratorTestCase;
    +import org.apache.hadoop.io.Text;
    +import org.junit.runners.Parameterized.Parameters;
    +
    +/**
    + * Framework tests for {@link WholeRowIterator}.
    + */
    +public class WholeRowIteratorTest extends BaseJUnit4IteratorTest {
    +
    +  @Parameters
    +  public static Object[][] parameters() {
    +    IteratorTestInput input = getIteratorInput();
    +    IteratorTestOutput output = getIteratorOutput();
    +    List<IteratorTestCase> tests = IteratorTestCaseFinder.findAllTestCases();
    +    return BaseJUnit4IteratorTest.createParameters(input, output, tests);
    +  }
    +
    +  private static final TreeMap<Key,Value> INPUT_DATA = createInputData();
    +  private static final TreeMap<Key,Value> OUTPUT_DATA = createOutputData();
    +
    +  private static TreeMap<Key,Value> createInputData() {
    +    TreeMap<Key,Value> data = new TreeMap<>();
    +
    +    data.put(new Key("1", "", "a"), new Value("1a".getBytes()));
    +    data.put(new Key("1", "", "b"), new Value("1b".getBytes()));
    +    data.put(new Key("1", "a", "a"), new Value("1aa".getBytes()));
    +    data.put(new Key("1", "a", "b"), new Value("1ab".getBytes()));
    +    data.put(new Key("1", "b", "a"), new Value("1ba".getBytes()));
    +
    +    data.put(new Key("2", "a", "a"), new Value("2aa".getBytes()));
    +    data.put(new Key("2", "a", "b"), new Value("2ab".getBytes()));
    +    data.put(new Key("2", "a", "c"), new Value("2ac".getBytes()));
    +    data.put(new Key("2", "c", "c"), new Value("2cc".getBytes()));
    +
    +    data.put(new Key("3", "a", ""), new Value("3a".getBytes()));
    +
    +    data.put(new Key("4", "a", "b"), new Value("4ab".getBytes()));
    +
    +    data.put(new Key("5", "a", "a"), new Value("5aa".getBytes()));
    +    data.put(new Key("5", "a", "b"), new Value("5ab".getBytes()));
    +    data.put(new Key("5", "a", "c"), new Value("5ac".getBytes()));
    +    data.put(new Key("5", "a", "d"), new Value("5ad".getBytes()));
    +
    +    data.put(new Key("6", "", "a"), new Value("6a".getBytes()));
    +    data.put(new Key("6", "", "b"), new Value("6b".getBytes()));
    +    data.put(new Key("6", "", "c"), new Value("6c".getBytes()));
    +    data.put(new Key("6", "", "d"), new Value("6d".getBytes()));
    +    data.put(new Key("6", "", "e"), new Value("6e".getBytes()));
    +    data.put(new Key("6", "1", "a"), new Value("61a".getBytes()));
    +    data.put(new Key("6", "1", "b"), new Value("61b".getBytes()));
    +    data.put(new Key("6", "1", "c"), new Value("61c".getBytes()));
    +    data.put(new Key("6", "1", "d"), new Value("61d".getBytes()));
    +    data.put(new Key("6", "1", "e"), new Value("61e".getBytes()));
    +
    +    return data;
    +  }
    +
    +  private static TreeMap<Key,Value> createOutputData() {
    +    TreeMap<Key,Value> data = new TreeMap<>();
    +
    +    Text row = null;
    +    List<Key> keys = new ArrayList<>();
    +    List<Value> values = new ArrayList<>();
    +
    +    // Generate the output data from the input data
    +    for (Entry<Key,Value> entry : INPUT_DATA.entrySet()) {
    --- End diff --
    
    Using [RowIterator](https://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/client/RowIterator.html)
may simplify this code a bit.


> create an iterator fuzz tester
> ------------------------------
>
>                 Key: ACCUMULO-626
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-626
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: test
>            Reporter: Eric Newton
>            Assignee: Josh Elser
>
> Users often write iterators without fully understanding its limits and lifetime. Accumulo
should have an iterator fuzz-tester which will take user data and run the iterator under extreme
conditions.  For example, it should re-create and re-seek the iterator with every key returned.
 It could automatically compare results of such a run with the naive run, which seeks to the
beginning and scans all the data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message