zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maarten Koopmans <maar...@vrijheid.net>
Subject Re: zkfuse
Date Tue, 24 Nov 2009 22:56:48 GMT
Hi Mahadev,

Quite simple: it will function as the locking/versioning back end for a 
custom WebDAV daemon (that stores its data in an internal cloud, i.e. 
Eucalyptus/Walrus).

So a user "joe" that has a file /foo/bar would translate to:

/(sha1(joe))/foo/bar  - file bar
/(sha1(joe))/foo/bar/versions - pointer to a file containing version 
history (sha1 pointers)
/(sha1(joe))/foo/bar/current  - file bar current version pointer, a sha1 
pointing directly to a file
/(sha1(joe))/foo/bar/__locks/ - dir containing locks for file bar in dir foo

Note that all files will be transparently encrypted and versioned and 
renamed to their sha1 - and then stored in an S3-like flat object space.

ZooKeeper reconstructs the filesystem layout + locking, and should be 
able to do that for lots (100,000s) of users. I am halfway a C binding 
(to REBOL, why REBOL? Shortest route to success for me.), but zkfuse is 
fast and functional enough (per WebDAV server instance). So I'll stick 
to zkfuse, I have so much other stuff to do. And I can always optimize 
in the background. The WebDAV server is custom written (in the process 
of being so...) and will provide the illusion of a user mountable 
"infinite drive". Locking is needed for Office, Mac to work.

The elasticity in the computing cloud makes the WebDAV daemons 
horizontally scalable as long as ZooKeeper can keep up, but with 40k 
writes/reads a second and a bit of clever caching that should be no 
problem. And we can always have multiple ensembles (what a great problem 
to have...) and do some clever tricks there with ensemble routing (get 
user joe always to ensemble #1).

So, there is the use case. Given that WebDAV is a good cross-platform 
user mountable file system, I hope to make a lot of people happy the 
next few months (I'd want it :-)

Regards,

Maarten

Mahadev Konar schreef:
> Hi Maarten,
>
>    zkfuse does not have any support for acls. We havent had much time to
> focus on zkfuse. Create/read/write/delete/ls are all supported. It was built
> mostly for infrequent updates and more of a browsing interface on
> filesystem. I don't think zkfuse is being used in production anywhere. Would
> you mind elaborating your use case?
>
> Thanks
> mahadev
>
>
> On 11/24/09 11:14 AM, "Maarten Koopmans"<maarten@vrijheid.net>  wrote:
>
>    
>> Hi,
>>
>> I just started using zkfuse, and this may very well suit my needs for
>> now. Thumbs up to the ZooKeeper team!
>>
>> What operations are supported (i.e. what is the best use of zkfuse). I
>> can see how files, dirs there creation and listing map quite nicely. ACLs?
>>
>> I have noticed two things on a fresk Ubuntu 9.10 (posting for future
>> archive reference):
>>
>> - I *have* to run in debug mode (-d)
>> - you have to add libboost  or it won't compile
>>
>> Regards,
>>
>> Maarten
>>      
>
>
>    


Mime
View raw message