curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Kanter <rkan...@cloudera.com>
Subject Re: Help with ACLProvider + Kerberos
Date Fri, 08 Nov 2013 01:46:26 GMT
I tried changing the line mentioned in CURATOR-58 to simply always set the
“ip” ACL that I was using in my test:
- zookeeper.create(subPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,
CreateMode.PERSISTENT);
+ zookeeper.create(subPath, new byte[0], Collections.singletonList(new
ACL(ZooDefs.Perms.ALL, new Id("ip", "127.0.0.1"))), CreateMode.PERSISTENT);

This caused my test to get stuck on trying to acquire the lock, which makes
me think the above change is working (presumably, ZK doesn’t think my IP is
“127.0.0.1”).  However, when I added a timeout to acquiring the lock
(i.e. writeLock.acquire(3,
TimeUnit.SECONDS);) so it could continue with the test, it still shows that
the znode has world ACLs.  So it doesn’t look like CURATOR-58 is the
problem :(

Any other ideas?

Would it be better for me to file a JIRA with my test code and continue the
conversation there?

thanks
- Robert



On Thu, Nov 7, 2013 at 4:29 PM, Robert Kanter <rkanter@cloudera.com> wrote:

> That could be it.  I’ll try to take a look and see if I can figure out a
> patch :)
>
> thanks
> - Robert
>
>
> On Thu, Nov 7, 2013 at 4:16 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> OK - I think this is the problem:
>>
>> https://issues.apache.org/jira/browse/CURATOR-58
>>
>> Curator is creating parent nodes without going through the ACLProvider.
>> Please watch CURATOR-58. If you can, please submit a patch.
>>
>> -JZ
>>
>> On Nov 7, 2013, at 3:51 PM, Robert Kanter <rkanter@cloudera.com> wrote:
>>
>> Are you sure that this is something my ACLProvider has to handle?  My
>> understanding is that the ACLProvider is a fairly “dumb” object that other
>> code is supposed to simply use to get the ACLs to apply when creating
>> znodes.  The ACLProvider interface only has two methods:
>> public List<ACL> getDefaultAcl();
>> public List<ACL> getAclForPath(String path);
>> So there’s nothing in here about protection mode.
>>
>> I think whatever code is changing the GUID-prefixed path into the
>> originally-named path (this does happen because when I browse the znodes
>> using ZKCli.sh it has “/foo” and not the prefixed one) needs to be checking
>> with the ACLProvider, and its not.  I tried looking around the Curator
>> code, but I wasn’t able to find what does this.  Do you know?
>>
>>
>> - Robert
>>
>>
>>
>> On Thu, Nov 7, 2013 at 3:40 PM, Jordan Zimmerman <
>> jordan@jordanzimmerman.com> wrote:
>>
>>> Yes, this is called "protection" mode. I'll paste the details from the
>>> Javadoc below. The TL;DR is that this is required when using
>>> sequential/ephemeral with ZooKeeper. So, your ACLProvider needs to be able
>>> to handle this.
>>>
>>> -Jordan
>>>
>>> from create().withProtection():
>>>
>>> It turns out there is an edge case that exists when creating
>>> sequential-ephemeral nodes. The creation can succeed on the server, but the
>>> server can crash before the created node name is returned to the client.
>>> However, the ZK session is still valid so the ephemeral node is not
>>> deleted. Thus, there is no way for the client to determine what node was
>>> created for them.
>>> Even without sequential-ephemeral, however, the create can succeed on
>>> the sever but the client (for various reasons) will not know it.
>>>
>>> Putting the create builder into protection mode works around this. The
>>> name of the node that is created is prefixed with a GUID. If node creation
>>> fails the normal retry mechanism will occur. On the retry, the parent path
>>> is first searched for a node that has the GUID in it. If that node is
>>> found, it is assumed to be the lost node that was successfully created on
>>> the first try and is returned to the caller.
>>>
>>> On Nov 7, 2013, at 1:13 PM, Robert Kanter <rkanter@cloudera.com> wrote:
>>>
>>> I did some more investigating on this issue.  It looks like when you
>>> acquire a lock, it creates the path by doing:
>>> ourPath =
>>> client.create().creatingParentsIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path);
>>> When I do that manually where path is “/foo”, it actually doesn’t
>>> create “/foo” yet and instead creates something like
>>> "/_c_2235fc7d-f5e8-4c3a-bb24-27c610022aaa-foo0000000000”.
>>>
>>> From what I can tell, this has to do with the withprotection() and
>>> EPHEMERAL_SEQUENTIAL mode and is to prevent some kind of problem I
>>> don’t quite understand from the javadocs.  In any case, I believe that
>>> something eventually must rename
>>> "/_c_2235fc7d-f5e8-4c3a-bb24-27c610022aaa-foo0000000000” to “/foo”.  When
I
>>> checked, "/_c_2235fc7d-f5e8-4c3a-bb24-27c610022aaa-foo0000000000” has the
>>> ACLs I set in the ACLProvider, so I’m thinking the problem must be
>>> happening when the znode is renamed.  I’m not sure where/when that happens,
>>> but I’d guess its not using the ACLProvider and that’s why “/foo" has
>>> the default ACLs.
>>>
>>> Do you think this is the cause?  Any idea on how to fix it or workaround
>>> it?
>>>
>>> thanks
>>> - Robert
>>>
>>>
>>>
>>> On Tue, Nov 5, 2013 at 1:58 PM, Robert Kanter <rkanter@cloudera.com>wrote:
>>>
>>>> I created the below test class using JUnit.  It starts a TestingServerand
connects to it; then it creates a path directly to verify that the
>>>> custom ACLProvider is being applied.  Then it tries to do the same
>>>> with an InterProcessReadWriteLock and fails the test because its using
>>>> the default ACLs.  I used “ip” instead of “sasl” to keep things simpler.
>>>>
>>>> I did take a quick look at the Curator code and it seemed to be using
>>>> the ACLProvider through the CuratorFramework when using locks, but
>>>> perhaps I missed something (and I’m not super familiar with the codebase).
>>>>
>>>> Please take a look; thanks!
>>>> - Robert
>>>>
>>>>
>>>> import java.util.Collections;
>>>> import java.util.List;
>>>> import junit.framework.TestCase;
>>>> import org.apache.curator.RetryPolicy;
>>>> import org.apache.curator.framework.CuratorFramework;
>>>> import org.apache.curator.framework.CuratorFrameworkFactory;
>>>> import org.apache.curator.framework.api.ACLProvider;
>>>>  import org.apache.curator.framework.recipes.locks.InterProcessMutex;
>>>> import
>>>> org.apache.curator.framework.recipes.locks.InterProcessReadWriteLock;
>>>> import org.apache.curator.retry.ExponentialBackoffRetry;
>>>> import org.apache.curator.test.TestingServer;
>>>> import org.apache.zookeeper.ZooDefs;
>>>> import org.apache.zookeeper.data.ACL;
>>>> import org.apache.zookeeper.data.Id<http://org.apache.zookeeper.data.id/>
>>>> ;
>>>>
>>>> public class TestLockACLs extends TestCase {
>>>>     private TestingServer zkServer;
>>>>     private CuratorFramework client;
>>>>     private final List<ACL> acls = Collections.singletonList(new
>>>> ACL(ZooDefs.Perms.ALL, new Id("ip", "127.0.0.1")));
>>>>
>>>>     @Override
>>>>     protected void setUp() throws Exception {
>>>>         super.setUp();
>>>>         zkServer = new TestingServer();
>>>>         createClient();
>>>>     }
>>>>
>>>>     @Override
>>>>     protected void tearDown() throws Exception {
>>>>         super.tearDown();
>>>>         client.close();
>>>>         zkServer.stop();
>>>>         zkServer.close();
>>>>     }
>>>>
>>>>     private void createClient() throws Exception {
>>>>         RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
>>>>         String zkConnectionString = zkServer.getConnectString();
>>>>         String zkNamespace = "ns";
>>>>         client = CuratorFrameworkFactory.builder()
>>>>                                             .namespace(zkNamespace)
>>>>
>>>> .connectString(zkConnectionString)
>>>>                                             .retryPolicy(retryPolicy)
>>>>                                             .aclProvider(new
>>>> MyACLProvider())
>>>>                                             .build();
>>>>         client.start();
>>>>     }
>>>>
>>>>     public void testLockACLs() throws Exception {
>>>>         // Create a path directly and verify that MyACLProvider is
>>>> being used
>>>>         client.create().forPath("/foo");
>>>>         assertNotNull(client.checkExists().forPath("/foo"));
>>>>         assertEquals(ZooDefs.Perms.ALL,
>>>> client.getACL().forPath("/foo").get(0).getPerms());
>>>>         assertEquals("ip",
>>>> client.getACL().forPath("/foo").get(0).getId().getScheme());
>>>>         assertEquals("127.0.0.1",
>>>> client.getACL().forPath("/foo").get(0).getId().getId());
>>>>
>>>>         // Now try creating a lock and we'll see that it incorrectly
>>>> has the default world ACLs
>>>>         // and doesn't seem to be using MyACLProvider
>>>>         InterProcessReadWriteLock lock = new
>>>> InterProcessReadWriteLock(client, "/bar");
>>>>         InterProcessMutex writeLock = lock.writeLock();
>>>>         writeLock.acquire();
>>>>         assertNotNull(client.checkExists().forPath("/bar"));
>>>>         assertEquals(ZooDefs.Perms.ALL,
>>>> client.getACL().forPath("/bar").get(0).getPerms());
>>>>         assertEquals("ip",
>>>> client.getACL().forPath("/bar").get(0).getId().getScheme());
>>>>         assertEquals("127.0.0.1",
>>>> client.getACL().forPath("/bar").get(0).getId().getId());
>>>>     }
>>>>
>>>>     public class MyACLProvider implements ACLProvider {
>>>>
>>>>         @Override
>>>>         public List<ACL> getDefaultAcl() {
>>>>             return acls;
>>>>         }
>>>>
>>>>         @Override
>>>>         public List<ACL> getAclForPath(String path) {
>>>>             return acls;
>>>>         }
>>>>     }
>>>> }
>>>>
>>>>
>>>> On Mon, Nov 4, 2013 at 6:10 PM, Jordan Zimmerman <
>>>> jordan@jordanzimmerman.com> wrote:
>>>>
>>>>> The ACLProvider should be called for every node created. It’s not
>>>>> getting called? Can you produce a test that shows this?
>>>>>
>>>>> -Jordan
>>>>>
>>>>> On Nov 4, 2013, at 5:57 PM, Robert Kanter <rkanter@cloudera.com>
>>>>> wrote:
>>>>>
>>>>> I have everything working now except for one thing:
>>>>> The ACLProvider doesn’t seem to be used for the locks (Curator’s
>>>>> InterProcessReadWriteLock); they are always created with the default
>>>>> fully open ACLs.  I know the ACLProvider is correct now because the
>>>>> service discovery is using it and znodes created by it have the correct
>>>>> ACLs.  InterProcessReadWriteLock’s constructor takes in the
>>>>> CuratorFramework object, which has the ACLProvider set.
>>>>>
>>>>> Any ideas?
>>>>> This sounds like it could be a Curator bug :(
>>>>> I’m not familiar with Curator’s codebase, but I’ll try to take
a look
>>>>> and see if I can figure it out.
>>>>>
>>>>> thanks
>>>>> - Robert
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Nov 4, 2013 at 1:09 PM, Robert Kanter <rkanter@cloudera.com>wrote:
>>>>>
>>>>>> I don’t have it 100% working yet, but I’ve figured out a lot
more, so
>>>>>> I thought I’d share in case anyone else runs into this:
>>>>>>
>>>>>> The ZooDefs.Ids.CREATOR_ALL_ACL predefined ACL that I was trying
to
>>>>>> use is for the “auth” scheme.  For SASL/Kerberos, we want “sasl”.
>>>>>>  The javadoc for the predefined one wasn’t very clear on that;
I had to
>>>>>> look at the code.  Using this is working:
>>>>>> Collections.singletonList(new ACL(Perms.ALL, new Id("sasl",
>>>>>> principal)));
>>>>>>
>>>>>> I was also able to find answers to the three questions I asked:
>>>>>> 1) Yes; looking through the code, its definitely grabbing the
>>>>>> ACLProvider and using it.
>>>>>> 2) Yes; I think the only way to do this is to recursively travel
>>>>>> through the znodes under /oozie and apply the ACL on starting up
Oozie.
>>>>>> We should only have to do this if previously it was setup without
>>>>>> security and has since been reconfigured to use security; so we should
only
>>>>>> have to do this once.  I can probably have a znode as a flag that
>>>>>> states if everything has ACLs or not to make it more efficient
>>>>>> 3) It doesn’t look like it; I’ll have to get the ZK client and
do it
>>>>>> from outside Curator
>>>>>>
>>>>>>
>>>>>> - Robert
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 28, 2013 at 5:47 PM, Jordan Zimmerman <
>>>>>> jordan@jordanzimmerman.com> wrote:
>>>>>>
>>>>>>> I don’t have any experience with this. Curator doesn’t do
much - it
>>>>>>> sets up the ACL as the CLI options dictate. I do know that you
also have to
>>>>>>> do work on the server side to make this work.
>>>>>>>
>>>>>>> -JZ
>>>>>>>
>>>>>>> On Oct 24, 2013, at 4:58 PM, Robert Kanter <rkanter@cloudera.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there any documentation on using an ACLProvider and/or Kerberos?
>>>>>>>
>>>>>>>
>>>>>>> From what I gathered at various sites, to use Kerberos, all I
have
>>>>>>> to do is set the following properties before building the CuratorFramework
>>>>>>> client:
>>>>>>> System.setProperty("java.security.auth.login.config",
>>>>>>> "/path/to/jaasConfFile");
>>>>>>>
>>>>>>> System.setProperty("zookeeper.authProvider.1","org.apache.zookeeper.server.auth.SASLAuthenticationProvider");
>>>>>>> System.setProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY,
>>>>>>> "Client");
>>>>>>> Looking at the logs for the client and server, this appears to
be
>>>>>>> working properly and my program is connecting to ZooKeeper using
Kerberos.
>>>>>>>
>>>>>>> The problem I'm having is with the ACLs.
>>>>>>>
>>>>>>> I'd like to set the ACLs so that only the Kerberos user running
the
>>>>>>> program can do anything.  From what I can tell, if I specify
an
>>>>>>> ACLProvider, then Curator will automatically use it for setting
>>>>>>> ACLs on all paths.  So, an ACLProvider like the following should
do
>>>>>>> what I want:
>>>>>>> public class CreatorACLProvider implements ACLProvider {
>>>>>>>    @Override
>>>>>>>     public List<ACL> getDefaultAcl() {
>>>>>>>         return ZooDefs.Ids.CREATOR_ALL_ACL;
>>>>>>>    }
>>>>>>>    @Override
>>>>>>>     public List<ACL> getAclForPath(String path) {
>>>>>>>         return ZooDefs.Ids.CREATOR_ALL_ACL;
>>>>>>>    }
>>>>>>> }
>>>>>>> Then I would just do this:
>>>>>>> client = CuratorFrameworkFactory.builder()
>>>>>>>                                 .namespace(zkNamespace)
>>>>>>>                                 .connectString(zkConnectionString)
>>>>>>>                                 .retryPolicy(retryPolicy)
>>>>>>>                                 .aclProvider(new
>>>>>>> CreatorACLProvider())
>>>>>>>                                 .build();
>>>>>>> client.start();
>>>>>>>
>>>>>>> However, this doesn't seem to be working.  The zkcli returns
this
>>>>>>> (on a newly created znode):
>>>>>>> [zk: localhost:2181(CONNECTED) 8] getAcl
>>>>>>> /oozie/locks/0000000-131024162150146-oozie-oozi-W
>>>>>>> 'world,'anyone
>>>>>>> : Cdr.
>>>>>>> Is there something that I missed?
>>>>>>>
>>>>>>> A few other questions:
>>>>>>> 1) Will the ACLProvider cause the ACLs to be applied to znodes
>>>>>>> created by the Curator recipes?  (e.g. InterProcessReadWriteLock,
>>>>>>> ServiceDiscovery, etc).  If not, then how should I go about setting
>>>>>>> the ACLs for these znodes?
>>>>>>> 2) I'm guessing that the ACLProvider is only applied when creating
>>>>>>> the znode, right; so existing znodes from before I added the
>>>>>>> ACLProvider won't have the ACLs I want, right?  What would be
the
>>>>>>> best way to apply the ACLs to any existing znodes that don't
have it set?
>>>>>>>  (My goal is to have all znodes under /oozie have the
>>>>>>> CREATOR_ALL_ACL)
>>>>>>> 3) Is there a way to set the ACLs on the namespace itself (i.e.
>>>>>>> /oozie)?  The methods that take a path (and automatically prepend
>>>>>>> the namespace) don't allow simply "/", so it seems like I'd have
to
>>>>>>> use the ZooKeeper client directly to set ACLs manually on the
namespace.
>>>>>>>  Or would simply passing an empty string "" work?
>>>>>>>
>>>>>>> thanks
>>>>>>> - Robert
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>

Mime
View raw message