hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator
Date Mon, 06 Jun 2016 17:59:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316885#comment-15316885
] 

Joseph commented on HBASE-15935:
--------------------------------

In general, the idea is to keep track of the linked-list loops that have been fully generated
and flushed inside of a HBase Table, "Flushed Table". When a circular linked list enters the
"Flushed Table" we can pass the circular linked list into a Walker job that will begin to
validate that circular linked list concurrently with Generation and/or Verify.

As of now I am considering a few possibilities for how this is going to be implemented:

1. Have Walk-Generation be a special mode inside of Loop. This will provide us a little more
flexibility in communicating between the Generation-Verify cycle and the Walk (eg: we could
pause or terminate Walker at certain times depending on whether Generation or Verification
is running). This is the version I have currently implemented, but my usage of Threads just
seems a little out of place in the IntegrationTestBigLinkedList code.
2. Modify Walker to have a special option, where it will only grab new nodes from the "Flushed
Table".
3. Another interesting option that I found was the concurrency option in the master GoraCI
project that would use a similar idea of "Flushed Table" to run Verifications concurrently
with Generation. I could also add this in as an optional Loop mode. 
See: https://github.com/keith-turner/goraci/commit/2bae337

Do you guys have any suggestions or comments on which of these approaches I should take? 

> Have a separate Walker task running concurrently with Generator   
> ------------------------------------------------------------------
>
>                 Key: HBASE-15935
>                 URL: https://issues.apache.org/jira/browse/HBASE-15935
>             Project: HBase
>          Issue Type: Sub-task
>          Components: integration tests
>            Reporter: Joseph
>            Assignee: Joseph
>            Priority: Minor
>
> Adding support for concurrent generation and verification. Will largely be copying over
the changes from 
> https://github.com/keith-turner/goraci/commit/9ae4323c3e3ae6bbc4746bcc313a97893c0368c9
> into the HBase version of the test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message