Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Mon, 9 Nov 2015 18:42:11 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12559679.1339070749000.18538.1447094531154@Atlassian.JIRA>
In-Reply-To: <JIRA.12559679.1339070749000@Atlassian.JIRA>
References: <JIRA.12559679.1339070749000@Atlassian.JIRA>
 <JIRA.12559679.1339070749578@arcas>
Subject: [jira] [Commented] (ACCUMULO-626) create an iterator fuzz tester
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997095#comment-14997095 ] 

ASF GitHub Bot commented on ACCUMULO-626:
-----------------------------------------

Github user wjsl commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/50#discussion_r44312285
  
    --- Diff: iterator-test-harness/src/main/java/org/apache/accumulo/iteratortest/IteratorTestUtil.java ---
    @@ -0,0 +1,44 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to you under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.accumulo.iteratortest;
    +
    +import java.util.Objects;
    +
    +import org.apache.accumulo.core.data.Key;
    +import org.apache.accumulo.core.data.Value;
    +import org.apache.accumulo.core.iterators.SortedKeyValueIterator;
    +import org.apache.accumulo.core.iterators.SortedMapIterator;
    +import org.apache.accumulo.core.iterators.system.ColumnFamilySkippingIterator;
    +import org.apache.accumulo.iteratortest.testcases.IteratorTestCase;
    +
    +/**
    + * A collection of methods that are helpful to the development of {@link IteratorTestCase}s.
    + */
    +public class IteratorTestUtil {
    +
    +  public static SortedKeyValueIterator<Key,Value> instantiateIterator(IteratorTestInput input) {
    +    try {
    +      return Objects.requireNonNull(input.getIteratorClass()).newInstance();
    +    } catch (InstantiationException | IllegalAccessException e) {
    +      throw new RuntimeException(e);
    +    }
    +  }
    +
    +  public static SortedKeyValueIterator<Key,Value> createSource(IteratorTestInput input) {
    +    return new ColumnFamilySkippingIterator(new SortedMapIterator(Objects.requireNonNull(input).getInput()));
    --- End diff --
    
    Might be good to make another iterator here (or wrapping the SMI) that re-uses the Key and Value instances returned by the getters. Storing the output of a source in a list or map without making a copy is a common pitfall.
    
    I had a small iterator benchmark that I modified to printout the identify of each Key and Value, and currently notice the same Value instance has the potential to be re-used. I will try to verify if Keys exhibit the same behavior-- I have heard anecdotes that they do.


> create an iterator fuzz tester
> ------------------------------
>
>                 Key: ACCUMULO-626
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-626
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: test
>            Reporter: Eric Newton
>            Assignee: Josh Elser
>
> Users often write iterators without fully understanding its limits and lifetime. Accumulo should have an iterator fuzz-tester which will take user data and run the iterator under extreme conditions.  For example, it should re-create and re-seek the iterator with every key returned.  It could automatically compare results of such a run with the naive run, which seeks to the beginning and scans all the data.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)