accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-14) Transition from full Accumulo mocking
Date Wed, 04 Apr 2012 01:14:22 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245948#comment-13245948
] 

Eric Newton commented on ACCUMULO-14:
-------------------------------------

I sat down last week with one goal: to be able to make all the services use a FileSystem object
provided by the test infrastructure, such that you could make a minicluster and do local testing
without needing an external HDFS instance.

That's when I discovered how much of the code just fetches a FileSystem instance using the
Hadoop configuration files.  It was doing pretty well until I ran into the HdfsZooInstance
class which is used to fetch server configuration information.

The cascading effects of passing FileSystem or Instance objects began touching a large percentage
of the code.  There are some parts, like a couple of the system Iterators, that expect to
be able to grab configuration information through global objects: there is no approved way
for Iterators be passed this kind of information.  Looking at the system configuration also
entails some security considerations. Anyhow, I fell so far down the rabbit hole, I reverted
the whole thing.

It's going to take a few attempts to make this happen.   I still hope to do it for 1.5.

In the mean time, the "Functional Tests" aren't looking so bad.  We can, and should, run them
more frequently.  I would like the map/reduce version to run nightly, at least.  Some of the
crazy tests that are in the Functional Tests should be added to the random walk tests.  Things
like "set the metadata tablet size 512 bytes and make a table with 5000 splits".  We need
to move from "verify normal operations work in the face of failures" to "normal is overrated."

                
> Transition from full Accumulo mocking
> -------------------------------------
>
>                 Key: ACCUMULO-14
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-14
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client
>            Reporter: Jesse Yates
>            Assignee: Eric Newton
>             Fix For: 1.5.0
>
>         Attachments: java_ACCUMULO-14-start-formatted.patch, java_ACCUMULO-14-start.patch
>
>
> Original discussion present here:http://mail-archives.apache.org/mod_mbox/incubator-accumulo-dev/201110.mbox/%3CCAGUtCHrN3p-7C8Uz7eT0Ze4L0ORzDj1f-x-JWdyrLc=Kx=o8ZQ@mail.gmail.com%3E
> Jesse Yates: "Maintaining a real instance and a mock instance ends up creating a lot
of extra work for dev's by ensuring that there impls are kept synchronized....[using a real]
instance of Accumulo running on the local file system. This would mean less work in the long
run to maintain a separate instance and better testing for users and new features."
> Keith Turner: "its very fast to create a mock instance which means that unit test run
very quickly...[and] its all in process which is useful for debugging...making mock leverage
more of the actual accumulo code, but possibly keep it in process for speed and debugging."
> This ticket is to track the discussion and development of a less mocked test instance
of accumulo, that relies more heavily on the actual implementation, rather than parallel mocking
features.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message