Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 6 Jun 2016 17:59:20 +0000 (UTC)
From: "Joseph (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12974933.1464815766000.3496.1465235960982@Atlassian.JIRA>
In-Reply-To: <JIRA.12974933.1464815766000@Atlassian.JIRA>
References: <JIRA.12974933.1464815766000@Atlassian.JIRA> <JIRA.12974933.1464815766060@arcas>
Subject: [jira] [Commented] (HBASE-15935) Have a separate Walker task
 running concurrently with Generator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 06 Jun 2016 17:59:22 -0000


    [ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316885#comment-15316885 ] 

Joseph commented on HBASE-15935:
--------------------------------

In general, the idea is to keep track of the linked-list loops that have been fully generated and flushed inside of a HBase Table, "Flushed Table". When a circular linked list enters the "Flushed Table" we can pass the circular linked list into a Walker job that will begin to validate that circular linked list concurrently with Generation and/or Verify.

As of now I am considering a few possibilities for how this is going to be implemented:

1. Have Walk-Generation be a special mode inside of Loop. This will provide us a little more flexibility in communicating between the Generation-Verify cycle and the Walk (eg: we could pause or terminate Walker at certain times depending on whether Generation or Verification is running). This is the version I have currently implemented, but my usage of Threads just seems a little out of place in the IntegrationTestBigLinkedList code.
2. Modify Walker to have a special option, where it will only grab new nodes from the "Flushed Table".
3. Another interesting option that I found was the concurrency option in the master GoraCI project that would use a similar idea of "Flushed Table" to run Verifications concurrently with Generation. I could also add this in as an optional Loop mode. 
See: https://github.com/keith-turner/goraci/commit/2bae337

Do you guys have any suggestions or comments on which of these approaches I should take? 

> Have a separate Walker task running concurrently with Generator   
> ------------------------------------------------------------------
>
>                 Key: HBASE-15935
>                 URL: https://issues.apache.org/jira/browse/HBASE-15935
>             Project: HBase
>          Issue Type: Sub-task
>          Components: integration tests
>            Reporter: Joseph
>            Assignee: Joseph
>            Priority: Minor
>
> Adding support for concurrent generation and verification. Will largely be copying over the changes from 
> https://github.com/keith-turner/goraci/commit/9ae4323c3e3ae6bbc4746bcc313a97893c0368c9
> into the HBase version of the test


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)