hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-11039) [VisibilityController] Integration test for labeled data set mixing and filtered excise
Date Wed, 09 Jul 2014 03:40:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055814#comment-14055814
] 

Andrew Purtell edited comment on HBASE-11039 at 7/9/14 3:38 AM:
----------------------------------------------------------------

If I run the test as a minicluster

{noformat}
$ mvn verify -Dit.test=IntegrationTestBigLinkedListWithVisibility -Dtest.output.tofile=false
{noformat}

the defaults for ITBLL mean we wait for a long time for jobs to finish. I didn't have time
to wait for the test to get past generation and copying to the excise and verification phases.
We should override the ITBLL command line parameters supplied when invoked by failsafe and
dial down the node count to something like 10,000 or 100,000. 

Looks like the table name for the test isn't overridden in all places:

{noformat}
2014-07-08 20:29:40,807 WARN  [Thread-670] policies.PeriodicRandomActionPolicy(61): Exception
occured during performing action: org.apache.hadoop.hbase.TableNotFoundException: IntegrationTestBigLinkedList
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:2686)
	at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:397)
	at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:402)
	at org.apache.hadoop.hbase.chaos.actions.ChangeVersionsAction.perform(ChangeVersionsAction.java:55)
	at org.apache.hadoop.hbase.chaos.policies.PeriodicRandomActionPolicy.runOneIteration(PeriodicRandomActionPolicy.java:59)
	at org.apache.hadoop.hbase.chaos.policies.PeriodicPolicy.run(PeriodicPolicy.java:41)
	at java.lang.Thread.run(Thread.java:745)
{noformat}




was (Author: apurtell):
If I run the test as a minicluster

{noformat}
$ mvn verify -Dit.test=IntegrationTestBigLinkedListWithVisibility -Dtest.output.tofile=false
{noformat}

the defaults for ITBLL mean we wait for a long time for jobs to finish. I didn't have time
to wait for the test to get past generation and copying to the excise and verification phases.
We should override the ITBLL command line parameters supplied when invoked by failsafe and
dial down the node count to something like 100,000. 

Looks like the table name for the test isn't overridden in all places:

{noformat}
2014-07-08 20:29:40,807 WARN  [Thread-670] policies.PeriodicRandomActionPolicy(61): Exception
occured during performing action: org.apache.hadoop.hbase.TableNotFoundException: IntegrationTestBigLinkedList
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:2686)
	at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:397)
	at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:402)
	at org.apache.hadoop.hbase.chaos.actions.ChangeVersionsAction.perform(ChangeVersionsAction.java:55)
	at org.apache.hadoop.hbase.chaos.policies.PeriodicRandomActionPolicy.runOneIteration(PeriodicRandomActionPolicy.java:59)
	at org.apache.hadoop.hbase.chaos.policies.PeriodicPolicy.run(PeriodicPolicy.java:41)
	at java.lang.Thread.run(Thread.java:745)
{noformat}



> [VisibilityController] Integration test for labeled data set mixing and filtered excise
> ---------------------------------------------------------------------------------------
>
>                 Key: HBASE-11039
>                 URL: https://issues.apache.org/jira/browse/HBASE-11039
>             Project: HBase
>          Issue Type: Test
>    Affects Versions: 0.98.1
>            Reporter: Andrew Purtell
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Critical
>             Fix For: 0.99.0, 1.0.0, 0.98.4
>
>         Attachments: HBASE-11039_ITBLL_v1.patch
>
>
> Create an integration test for the VisibilityController that:
> 1. Create several tables of test data
> 2. Assign a set of auths to each table. Label all entries in the table with appropriate
visibility expressions. Insure that some data in every table overlaps with data in other tables
at common row/family/qualifier coordinates. Generate data like ITBLL so we can verify all
data present later.
> 3. Mix the data from the different tables into a new common table
> 4. Verify for each set of auths defined in step #2 that all entries found in the source
table can be found in the common table. Like the ITBLL verification step but done N times
for each set of auths defined in step #2.
> 5. Choose one of the source tables. Get its set of auths. Perform a deletion with visibility
expression from the common table using those auths.
> 6. Verify that no data in the common table with the auth set chosen in #5 remains. A
simple row count with the set of auths chosen in #5 that should return 0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message