hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: ACL question w/ Zookeeper 3.1.1
Date Mon, 21 Sep 2009 23:13:55 GMT
Todd, I spent some time looking at your output and honestly I'm having 
trouble making sense of what you are saying. What's the diff btw v3 & 
v4? I'm afraid here are too many variables, can you help nail things down?

1) create a jira for this
https://issues.apache.org/jira/browse/ZOOKEEPER

2) if at all possible attach the code you are running that has problems, 
seems like you've boiled it down to a case where it is deterministic, 
this would be the best for us to debug. If you can't attach the code 
then include snippets - in particular the addAuthInfo call (w/parameter 
details) for your clients, and the individual create calls, including 
the acl specifics - and describe what your client(s) are doing in detail 
so that we can attempt to reproduce.

3) attach a trace level log from both the server and client during your 
test run, point out the time index when you see the auth failure.


btw, you might try doing a getACL(path...) just before the operation 
that's failing - it will give you some insight into what the acl is set 
to for that node.

Patrick

Todd Greenwood wrote:
> Patrick / Mahadev,
> 
> I've spent the last couple of days attempting to isolate this issue, and
> this is what I've come up with...
> 
> Mahadev's simple use case works fine, as posted. However, my more
> involved use cases are consistently failing w/ InvalidACL exceptions
> when I use digest authentication with Ids.CREATOR_ALL_ACL:
> 
> java.lang.Exception:
> com.audiencescience.util.zookeeper.wrapper.ZooWrapperException:
> org.apache.zookeeper.KeeperException$InvalidACLException:
> KeeperErrorCode = InvalidACL for /ACLTest
> 
> Prior to throwing this exception, the response is
> (Zookeeper.java:create()):
> r = {org.apache.zookeeper.proto.ReplyHeader@1445}"0,0,-112\n
> <mailto:{org.apache.zookeeper.proto.ReplyHeader@1445}> ". More debug
> data below.
> 
> So, while I can get Mahadev's simple example to work, I cannot get a
> more involved use case to work correctly. However, if I change my code
> to use Ids.OPEN_ACL_UNSAFE, then everything works fine. Example debug
> output below at "v5".
> 
> Could someone point me at non-trivial test cases for ACLs, and perhaps
> give me some insight into how to debug this issue further?
> 
> -Todd
> 
> 
> -------------------------------
> Code Snippet ZooKeeper.java
> -------------------------------
> 
>     public String create(String path, byte data[], List<ACL> acl,
>             CreateMode createMode)
>         throws KeeperException, InterruptedException
>     {
>         validatePath(path);
> 
>         RequestHeader h = new RequestHeader();
>         h.setType(ZooDefs.OpCode.create);
>         CreateRequest request = new CreateRequest();
>         CreateResponse response = new CreateResponse();
>         request.setData(data);
>         request.setFlags(createMode.toFlag());
>         request.setPath(path);
>         if (acl != null && acl.size() == 0) {
>             throw new KeeperException.InvalidACLException();
>         }
>         request.setAcl(acl);
>         ReplyHeader r = cnxn.submitRequest(h, request, response, null);
> 
>>>> v3
>>>> v5
>         if (r.getErr() != 0) {
> 
>>>> v4
>             throw
> KeeperException.create(KeeperException.Code.get(r.getErr()),
>                     path);
>         }
>         return response.getPath();
>     }
> 
> 
> ---------------------------------------------
> v3
> ---------------------------------------------
> this = {org.apache.zookeeper.ZooKeeper@1267}
> watchManager = {org.apache.zookeeper.ZooKeeper$ZKWatchManager@1379}
> state = {org.apache.zookeeper.ZooKeeper$States@1380}"CONNECTED"
> cnxn = {org.apache.zookeeper.ClientCnxn@1381}"sessionId:
> 0x123de5b3b1b0000\nlastZxid: 1\nxid: 2\nnextAddrToTry: 0\nserverAddrs:
> /127.0.0.1:2181\n"
> serverAddrs = {java.util.ArrayList@1386} size = 1
> authInfo = {java.util.ArrayList@1387} size = 1
> [0] = {org.apache.zookeeper.ClientCnxn$AuthData@1398}
> scheme = {java.lang.String@1244}"digest"
> data = {byte[13]@1399}
> pendingQueue = {java.util.LinkedList@1388} size = 0
> outgoingQueue = {java.util.LinkedList@1389} size = 1
> nextAddrToTry = 0
> connectTimeout = 40000
> readTimeout = 26666
> sessionTimeout = 50000
> zooKeeper = {org.apache.zookeeper.ZooKeeper@1267}
> watcher = {org.apache.zookeeper.ZooKeeper$ZKWatchManager@1379}
> sessionId = 82153701637816320
> sessionPasswd = {byte[16]@1390}
> sendThread =
> {org.apache.zookeeper.ClientCnxn$SendThread@1259}"Thread[main-SendThread
> ,5,main]"
> eventThread =
> {org.apache.zookeeper.ClientCnxn$EventThread@1266}"Thread[main-EventThre
> ad,5,main]"
> selector = {sun.nio.ch.EPollSelectorImpl@1391}
> closing = false
> eventOfDeath = {java.lang.Object@1392}
> lastZxid = 1
> xid = 2
> acl = {java.util.ArrayList@1242} size = 1
> [0] = {org.apache.zookeeper.data.ACL@1348}"31,s{'auth,'}\n"
> perms = 31
> id = {org.apache.zookeeper.data.Id@1404}"'auth,'\n"
> request =
> {org.apache.zookeeper.proto.CreateRequest@1360}"'/ACLTest,,v{},0\n"
> path = {java.lang.String@1314}"/ACLTest"
> data = {byte[0]@1339}
> acl = null
> flags = 0
> path = {java.lang.String@1314}"/ACLTest"
> 
> 
> ---------------------------------------------
> v4
> ---------------------------------------------
> this = {org.apache.zookeeper.ZooKeeper@1267}
> watchManager = {org.apache.zookeeper.ZooKeeper$ZKWatchManager@1379}
> state = {org.apache.zookeeper.ZooKeeper$States@1453}"CLOSED"
> cnxn = {org.apache.zookeeper.ClientCnxn@1381}"sessionId:
> 0x123de5b3b1b0000\nlastZxid: 1\nxid: 3\nnextAddrToTry: 0\nserverAddrs:
> /127.0.0.1:2181\n"
> serverAddrs = {java.util.ArrayList@1386} size = 1
> authInfo = {java.util.ArrayList@1387} size = 1
> [0] = {org.apache.zookeeper.ClientCnxn$AuthData@1398}
> scheme = {java.lang.String@1244}"digest"
> data = {byte[13]@1399}
> pendingQueue = {java.util.LinkedList@1388} size = 0
> outgoingQueue = {java.util.LinkedList@1389} size = 0
> nextAddrToTry = 0
> connectTimeout = 40000
> readTimeout = 26666
> sessionTimeout = 50000
> zooKeeper = {org.apache.zookeeper.ZooKeeper@1267}
> watcher = {org.apache.zookeeper.ZooKeeper$ZKWatchManager@1379}
> sessionId = 82153701637816320
> sessionPasswd = {byte[16]@1390}
> sendThread =
> {org.apache.zookeeper.ClientCnxn$SendThread@1259}"Thread[main-SendThread
> ,5,]"
> eventThread =
> {org.apache.zookeeper.ClientCnxn$EventThread@1266}"Thread[main-EventThre
> ad,5,main]"
> selector = {sun.nio.ch.EPollSelectorImpl@1391}
> closing = false
> eventOfDeath = {java.lang.Object@1392}
> lastZxid = 1
> xid = 3
> response = {org.apache.zookeeper.proto.CreateResponse@1365}"\n"
> r = {org.apache.zookeeper.proto.ReplyHeader@1445}"0,0,-112\n"
> request =
> {org.apache.zookeeper.proto.CreateRequest@1360}"'/ACLTest,,v{s{31,s{'aut
> h,'}}},0\n"
> path = {java.lang.String@1314}"/ACLTest"
> data = {byte[0]@1339}
> acl = {java.util.ArrayList@1242} size = 1
> flags = 0
> path = {java.lang.String@1314}"/ACLTest"
> h = {org.apache.zookeeper.proto.RequestHeader@1352}"2,1\n"
> cnxn = {org.apache.zookeeper.ClientCnxn@1381}"sessionId:
> 0x123de5b3b1b0000\nlastZxid: 1\nxid: 3\nnextAddrToTry: 0\nserverAddrs:
> /127.0.0.1:2181\n"
> 
> 
> --------------------------------------
> v5
> 
> NOTE: If I use Ids.OPEN_ACL_UNSAFE, then everything works fine. Here's
> an example of the debug state after a create()...
> --------------------------------------
> 
> this = {org.apache.zookeeper.ZooKeeper@1266}
> watchManager = {org.apache.zookeeper.ZooKeeper$ZKWatchManager@1397}
> state = {org.apache.zookeeper.ZooKeeper$States@1398}"CONNECTED"
> cnxn = {org.apache.zookeeper.ClientCnxn@1374}"sessionId:
> 0x123de6ba8de0000\nlastZxid: 2\nxid: 3\nnextAddrToTry: 0\nserverAddrs:
> /127.0.0.1:2181\n"
> serverAddrs = {java.util.ArrayList@1403} size = 1
> authInfo = {java.util.ArrayList@1404} size = 1
> [0] = {org.apache.zookeeper.ClientCnxn$AuthData@1415}
> scheme = {java.lang.String@1244}"digest"
> data = {byte[13]@1416}
> pendingQueue = {java.util.LinkedList@1405} size = 0
> outgoingQueue = {java.util.LinkedList@1406} size = 0
> nextAddrToTry = 0
> connectTimeout = 40000
> readTimeout = 26666
> sessionTimeout = 50000
> zooKeeper = {org.apache.zookeeper.ZooKeeper@1266}
> watcher = {org.apache.zookeeper.ZooKeeper$ZKWatchManager@1397}
> sessionId = 82153772198789120
> sessionPasswd = {byte[16]@1407}
> sendThread =
> {org.apache.zookeeper.ClientCnxn$SendThread@1259}"Thread[main-SendThread
> ,5,main]"
> eventThread =
> {org.apache.zookeeper.ClientCnxn$EventThread@1265}"Thread[main-EventThre
> ad,5,main]"
> selector = {sun.nio.ch.EPollSelectorImpl@1408}
> closing = false
> eventOfDeath = {java.lang.Object@1409}
> lastZxid = 2
> xid = 3
> response = {org.apache.zookeeper.proto.CreateResponse@1360}"'/ACLTest\n"
> r = {org.apache.zookeeper.proto.ReplyHeader@1389}"2,2,0\n"
> xid = 2
> zxid = 2
> err = 0
> request =
> {org.apache.zookeeper.proto.CreateRequest@1355}"'/ACLTest,,v{s{15,s{'wor
> ld,'anyone}}},0\n"
> path = {java.lang.String@1314}"/ACLTest"
> h = {org.apache.zookeeper.proto.RequestHeader@1347}"2,1\n"
> cnxn = {org.apache.zookeeper.ClientCnxn@1374}"sessionId:
> 0x123de6ba8de0000\nlastZxid: 2\nxid: 3\nnextAddrToTry: 0\nserverAddrs:
> /127.0.0.1:2181\n"
> 
>> -----Original Message-----
>> From: Todd Greenwood [mailto:toddg@audiencescience.com]
>> Sent: Friday, September 18, 2009 11:27 AM
>> To: Patrick Hunt; zookeeper-dev@hadoop.apache.org; zookeeper-
>> user@hadoop.apache.org
>> Subject: RE: ACL question w/ Zookeeper 3.1.1
>>
>> Patrick / Mahadev,
>>
>> Thanks for the heads-up!
>>
>> Apparently I *am* receiving email from zookeeper-user but it is being
>> filtered out as spam. This just started happening, but I'll rectify on
>> my end.
>>
>> I'm working thru Mahadev's response and will respond shortly (and
> search
>> for other postings, as well). Appologies for the cross post.
>>
>> -Todd
>>
>>> -----Original Message-----
>>> From: Patrick Hunt [mailto:phunt@apache.org]
>>> Sent: Friday, September 18, 2009 11:19 AM
>>> To: zookeeper-dev@hadoop.apache.org;
> zookeeper-user@hadoop.apache.org
>>> Cc: Todd Greenwood
>>> Subject: Re: ACL question w/ Zookeeper 3.1.1
>>>
>>> Todd, there were other responses as well. Are you seeing other
> traffic
>>> from the lists? (perhaps a spam filtering issue?)
>>>
>>> Patrick
>>>
>>> Mahadev Konar wrote:
>>>> HI todd,
>>>>   We did respond on zookeeper-user. Here is my response in case
> you
>>> didn't
>>>> see it...
>>>>
>>>>
>>>> HI todd,
>>>>  From what I understand, you are sayin that a creator_all_acl does
>> not
>>> work
>>>> with auth?
>>>>
>>>>  I tried the following with CREATOR_ALL_ACL and it seemed to work
>> for
>>> me...
>>>> import org.apache.zookeeper.CreateMode;
>>>> import org.apache.zookeeper.WatchedEvent;
>>>> import org.apache.zookeeper.Watcher;
>>>> import org.apache.zookeeper.ZooKeeper;
>>>> import org.apache.zookeeper.data.ACL;
>>>> import org.apache.zookeeper.ZooDefs.Ids;
>>>> import java.util.ArrayList;
>>>> import java.util.List;
>>>>
>>>> public class TestACl implements Watcher {
>>>>
>>>>     public static void main(String[] argv) throws Exception {
>>>>         List<ACL> acls = new ArrayList<ACL>(1);
>>>>         String authentication_type = "digest";
>>>>         String authentication = "mahadev:some";
>>>>
>>>>         for (ACL ids_acl : Ids.CREATOR_ALL_ACL) {
>>>>             acls.add(ids_acl);
>>>>         }
>>>>         TestACl tacl = new TestACl();
>>>>         ZooKeeper zoo = new ZooKeeper("localhost:2181", 3000,
> tacl);
>>>>         zoo.addAuthInfo(authentication_type,
>> authentication.getBytes());
>>>>         zoo.create("/some", new byte[0], acls,
>> CreateMode.PERSISTENT);
>>>>         zoo.setData("/some", new byte[0], -1);
>>>>     }
>>>>
>>>>     @Override
>>>>     public void process(WatchedEvent event) {
>>>>
>>>>
>>>>     }
>>>> }
>>>>
>>>>
>>>> And it worked on my set of zookeeper servers....
>>>>
>>>> And then
>>>> I tried
>>>>
>>>> Without auth
>>>>
>>>> Getdata("/some")
>>>>
>>>> Which correctly gave me the error:
>>>>
>>>>
>>>> Exception in thread "main"
>>>> org.apache.zookeeper.KeeperException$NoAuthException:
>> KeeperErrorCode =
>>>> NoAuth for /some
>>>>     at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:104)
>>>>     at
>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>>>>     at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
>>>>     at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921)
>>>>     at
>>>>
> org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:692)
>>>>     at
> org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:579)
>>>>     at
>>>>
>> org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:351)
>>>>     at
>> org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:309)
>>>>     at
>> org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:268)
>>>>
>>>> Is this what you are trying to do?
>>>>
>>>> Thanks
>>>> mahadev
>>>>
>>>>
>>>> On 9/18/09 10:33 AM, "Todd Greenwood" <toddg@audiencescience.com>
>> wrote:
>>>>> Appologies for cross posting, but I haven't received a response
> on
>>> this.
>>>>> Quite simply, could someone point me to a working
>> example/tutorial/docs
>>>>> that describe how to use digest ACLs in zookeeper 3.1.1? The docs
>> that
>>> I
>>>>> have found (referenced below) have not clarified this for me.
>>>>>
>>>>> -Todd
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Todd Greenwood
>>>>>> Sent: Thursday, September 17, 2009 5:05 PM
>>>>>> To: 'zookeeper-user@hadoop.apache.org'
>>>>>> Subject: ACL question w/ Zookeeper 3.1.1
>>>>>>
>>>>>> I'm attempting to secure a zookeeper installation using
> zookeeper
>>>>> ACLs.
>>>>>> However, I'm finding that while Ids.OPEN_ACL_UNSAFE works great,
>> my
>>>>>> attempts at using Ids.CREATOR_ALL_ACL are failing. Here's a code
>>>>> snippet:
>>>>>> public class ZooWrapper
>>>>>> {
>>>>>>
>>>>>> /*
>>>>>> 1. Here I'm setting up my authentication. I've got an ACL list,
>> and my
>>>>>> authentication strings.
>>>>>> */
>>>>>>     private final List<ACL> acl = new ArrayList<ACL>(
1 );
>>>>>>     private static final String authentication_type = "digest";
>>>>>>     private static final String authentication =
>>>>> "audiencescience:gravy";
>>>>>>     public ZooWrapper( final String connection_string,
>>>>>>                        final String path,
>>>>>>                        final int connectiontimeout ) throws
>>>>>> ZooWrapperException
>>>>>>     {
>>>>>> ...
>>>>>> /*
>>>>>> 2. Here I'm adding the acls
>>>>>> */
>>>>>>
>>>>>>         // This works (creates nodes, sets data on nodes)
>>>>>>         for ( ACL ids_acl : Ids.OPEN_ACL_UNSAFE )
>>>>>>         {
>>>>>>             acl.add( ids_acl);
>>>>>>         }
>>>>>>
>>>>>> /*
>>>>>> NOTE:  This does not work (nodes are not created, cannot set
> data
>> on
>>>>> nodes
>>>>>> b/c nodes do not exist)
>>>>>> */
>>>>>>
>>>>>> //        for ( ACL ids_acl : Ids.CREATOR_ALL_ACL )
>>>>>> //        {
>>>>>> //            acl.add( ids_acl );
>>>>>> //        }
>>>>>>
>>>>>> /*
>>>>>> 3. Finally, I create a new zookeeper instance and add my
>> authorization
>>>>>> info to it.
>>>>>> */
>>>>>>      zoo = new ZooKeeper( connection_string, connectiontimeout,
>> this
>>>>> );
>>>>>>      zoo.addAuthInfo( authentication_type,
>> authentication.getBytes() )
>>>>>> /*
>>>>>> 4. Later, I try to write some data into zookeeper by first
>> creating
>>>>> the
>>>>>> node, and then calling setdata...
>>>>>> */
>>>>>>       zoo.create( path, new byte[0], acl, CreateMode.PERSISTENT
> );
>>>>>>       zoo.setData( path, bytes, -1 )
>>>>>>
>>>>>> As I mentioned above, when I add Ids.OPEN_ACL_UNSAFE to acl,
> then
>> both
>>>>> the
>>>>>> create and setData succeed. However, when I use
>> Ids.CREATOR_ALL_ACL,
>>>>> then
>>>>>> the nodes are not created. Am I missing something obvious w/
>> respect
>>>>> to
>>>>>> configuring ACLs?
>>>>>>
>>>>>> I've used the following references:
>>>>>>
>>>>>>
> http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperProgrammers.html
>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-
>>>>>>
>> commits/200807.mbox/%3C20080731201025.C62092388873@eris.apache.org%3E
>>>>>> http://books.google.com/books?id=bKPEwR-
>>>>>>
> Pt6EC&pg=PT404&lpg=PT404&dq=zookeeper+ACL+digest+%22new+Id%22&source=bl&
>>>>> ot
>>>>>
> s=kObz0y8eFk&sig=VFCAsNW0mBJyZswoweJDI31iNlo&hl=en&ei=Z82ySojRFsqRlAeqxs
>>>>> yI
>>>>>
> Dw&sa=X&oi=book_result&ct=result&resnum=6#v=onepage&q=zookeeper%20ACL%20
>>>>> di
>>>>>> gest%20%22new%20Id%22&f=false
>>>>>>
>>>>>> -Todd
> 

Mime
View raw message