hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Reed <br...@yahoo-inc.com>
Subject RE: setACL semantics
Date Wed, 11 Feb 2009 04:00:12 GMT
you may not have noticed, but in the prep request processor we transform zookeeper requests
into idempotent transactions. this helps alot for some of our recovery scenarios, but to pull
it off we have to be able to look into the future and figure out what the state of the system
will look like when the transaction executes. remember we could have hundreds if not thousands
of requests in flight. so, for the requests that change state, we must do all the checks ahead
of time in the prep request processor by looking into the future. check out the following:

"/foo" has "v0" version 1 with open acl.

If we get a pipeline of requests that all get through the prep request processor before the
first requests hits the final request processor, what should happen? The pipelining must be
completely transparent. Everything must act as if the requests are performed one by one without
any processing.

1) setData("/foo", "v1", 1);
2) setData("/foo", "v2", -1);
3) setData("/foo", "v3", 1);
4) setACL("/foo", restricted permission);
5) setData("/foo", "v4", -1);

Step 1) this operation succeeds. "/foo" will be set to "v1" version 2.
Step 2) this operation succeeds. "/foo" will be set to "v2" version 3.
Step 3) this operation fails. even though /foo is still at version 1 in the data tree, we
have the look ahead list that shows that when 3) executes /foo will be at version 3, so we
fail due to a bad version
Step 4) this operation succeeds. /foo now has a restricted permission.
Step 5) this operation fails. even though /foo still has open acl in the data tree, we have
the look ahead list that shows that when 5) executes /foo will have a restricted permission.

so in the end /foo will have "v2" version 3 with a restricted permission, which would be exactly
the same as if the requests were not pipelined.

no waiting, no problem.

because we stage the result of the execution in the prep request processor, we have to do
the check there. otherwise, we run into major race conditions.

ben

________________________________________
From: Manos Kapritsos [manos@cs.utexas.edu]
Sent: Tuesday, February 10, 2009 6:30 PM
To: zookeeper-dev@hadoop.apache.org
Subject: Re: setACL semantics

You are right about the getData, and it was probably a wrong choice.
Both the getData and getChildren have an extra ACL check in the
FinalRequestProcessor. My question is pretty much this: shouldn't the
other request types have such a check as well? What if client B has
issued a setData()? setData is only ACL-checked in PrepRequestProcessor,
so it may be the case that it succeeds, while client A's the setAcl()
has been ordered to be executed first.

It seems that an ACL check at the FinalRequestProcessor for most request
types would solve this issue. As it is there only for getData and
getChildren, I wonder why that is and if scenarios as the one above are
considered important.

By the way, does Zookeeper require clients to wait for the reply for
setAcl()? If not, then the "single client, single server" model would
face the problem even more obviously.

Thank you for the replies,
Manos

Benjamin Reed wrote:
> Manos,
>
> in your scenario the getData will not succeed. the acl will be checked in the FinalRequestProcessor
after the setAcl has passed through the final request processor. execution only happens in
the final request processor and always happens in order.
>
> the only way a getData will succeed after a setAcl succeeds if it is done on a lagging
follower, but in that case it will see a lagging view of the system. consider the following:
>
> /foo has an open ACL
>
> client A: setAcl("/foo", restricted ACL); setData("/foo", new data);
> client B: getData("/foo");
>
> no matter what happens, what the delays are, if B is faster or slower than A, client
B will not see "new data". it will either pass the ACL check and see the old data or it will
fail with a permission exception.
>
> ben
> ________________________________________
> From: Manos Kapritsos [manos@cs.utexas.edu]
> Sent: Tuesday, February 10, 2009 4:34 PM
> To: zookeeper-dev@hadoop.apache.org
> Subject: Re: setACL semantics
>
> More or less, yes.
> If the requests are from the same client, then you say that there will
> not be a problem? I guess that is true if you always wait for the
> response of the first request in order to execute the second. I am not
> sure if that is a requirement for all Zookeeper client implementations.
> As for two different clients (which was the case I was thinking about),
> this seems to be a problem. I will agree that (if clients only have one
> outstanding request) the two requests are concurrent and either order of
> execution is considered to be acceptable, but it could be that two
> different replicas receive the two requests in the same order, but
> effectively execute them in a different order. In any case, it feels
> wrong (at least to me) that a getData would succeed when a setACL that
> prevents it has already been accepted to be processed.
>
> Manos
>
> Mahadev Konar wrote:
>
>> Hi Manos,
>>  If the setAcl and getdata are from the same client then they are all
>> handled in order. So you would get an unauthroized exception when you do a
>> getdata.
>>
>>
>> If two diff clients do setacl and getdata it might be that the getdata in
>> your case will succeed before the setacl returns on the first client.
>>
>> Is that what you meant?
>>
>> mahadev
>>
>>
>> On 2/10/09 2:15 PM, "Manos Kapritsos" <manos@cs.utexas.edu> wrote:
>>
>>
>>
>>> Hi all,
>>>
>>> I have a question about the way setACL functions. It seems that the
>>> PreRequestProcessor handles all kinds of requests the same, checks the
>>> validity of the corresponding ACL, and enqueues them to Sync and Final
>>> processors. Maybe I am missing something here, but this behaviour seems
>>> weird. What if a setACL request comes, setting the ACL of a path (e.g. /
>>> ) to an IP (e.g. 128.62.32.32) , instead of its old value (e.g. World).
>>> This request will pass the ACL check, and will be enqueued to be
>>> processed by the next processors. Assume that the next request is a
>>> getData("/") from an IP other than 128.62.32.32. If this request is
>>> processed by the PreRequestProcessor before the setACL request is
>>> processed by the FinalRequestProcessor, then it will pass the ACL check
>>> (which it should not, since it came after the setACL request). It seems
>>> that there is a race condition here that should not exist.
>>>
>>> Let me know if this is actually the case or I am missing something. I am
>>> using version 3.0.1 of the code.
>>>
>>> Thank you,
>>> Manos
>>>
>>>


Mime
View raw message