accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: [DISCUSS] MiniAccumuloCluster goals and approach
Date Wed, 26 Mar 2014 20:21:15 GMT
Yes, very much experimental at this point.

What I'm most concerned about is having reasonable hooks up front, not 
trying to make an implementation for inclusion 1.6.0.

Regarding additions, the implementations already contains most things I 
would want to expose. I haven't come up with anything that would be 
generally returned through the "API" rather than through this proposed 
implementation (e.g. YARN connection information)

On 3/26/14, 11:57 AM, Keith Turner wrote:
> What you are trying to do sounds interesting.  It also sounds experimental
> and in the early stages.   Is there anything specific you think should be
> done for 1.6.0 w/ regards to MAC API?
>
>
> On Wed, Mar 26, 2014 at 2:26 PM, Josh Elser <josh.elser@gmail.com> wrote:
>
>> On 3/26/14, 11:13 AM, Keith Turner wrote:
>>
>>> On Wed, Mar 26, 2014 at 2:05 PM, Josh Elser <josh.elser@gmail.com> wrote:
>>>
>>>   On 3/26/14, 10:57 AM, Keith Turner wrote:
>>>>
>>>>   Can you give an example of what you are thinking of? I don't understand
>>>>> you
>>>>> viewpoint either
>>>>>
>>>>>
>>>> Sure. One limitation of MAC, in general as a testing harness, is that it
>>>> doesn't adequately exercise multi-node implementations. You can run
>>>> multiple tservers, but they are all on the same host which limits the
>>>> validity of a "robust" test. This is my immediate goal.
>>>>
>>>> Multi-node deployments are capable using something like Mesos or Yarn.
>>>> Given that there is already functioning support to deploy Accumulo on
>>>> Yarn,
>>>> this was my goal.
>>>>
>>>> My goal is to be able to have the ability to run all of our AbstractMacIT
>>>> implementations against "real" hardware without changing a single line of
>>>> test code (ok - maybe a line or two to do injection of the MAC
>>>> implementation). The point is, I believe there could be a huge testing
>>>> gain
>>>> from being able to write tests which leverage yarn, have the same
>>>> programmatic configuration API from MAC, and provide near "real" Accumulo
>>>> semantics.
>>>>
>>>>
>>> Ok so you want to MAC to be an interface so that you can provide a
>>> completely different implementation?
>>>
>>>
>> Correct. Some things would serve well in a common abstract base (e.g.
>> numTservers, siteXml configuration), but all the nonsense about creating
>> directory structures and managing Processes is implementation specific.
>>
>> Perhaps I could create a new interface that the current implementation
>> implements which still provides the same semantics from 1.4 and 1.5. Let me
>> see if I can mock up what I'm thinking -- that will probably be easier than
>> me trying to write it out.
>>
>

Mime
View raw message