zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tj5527 <tj5...@protonmail.com>
Subject Re: Write Ahead Log
Date Fri, 12 Jan 2018 15:54:04 GMT
Files are source files e.g. FileTxnLog.java that is related to Write Ahead Log and snapshot
mechanism. And thanks for the explanation!

> -------- Original Message --------
> Subject: Re: Write Ahead Log
> Local Time: January 9, 2018 7:07 PM
> UTC Time: January 9, 2018 6:07 PM
> From: afine@apache.org
> To: user@zookeeper.apache.org
>
> I'm not 100% sure what you mean by "what files can be checked". I think you are looking
for where the write ahead log and snapshot are stored. This directory (optionally directories)
are specified in the configuration file.
>
> First, there is the dataDir, which, according to the documentation is "the location to
store the in-memory database snapshots and, unless specified otherwise, the transaction log
of updates to the database".
>
> The "unless specified otherwise" refers to the optional configuration dataLogDir. The
dataDir always contains the snapshots in "snapshot.X" files and if dataLogDir is not specified
log files in "log.X" (X is a value that ZooKeeper uses for keeping track of transactions).
If dataLogDir is specified, dataDir will only contain snapshots and dataLogDir will have the
log files.
>
> Small note, on ZooKeeper 3.4.11 there is a bug regarding dataLogDir not functioning properly
(see: https://issues.apache.org/jira/browse/ZOOKEEPER-2960). This does not exist prior to
3.4.11 and should be fixed in the next release.
>
> Thanks,
> Abe
>
> On Tue, Jan 9, 2018, at 01:29, tj5527 wrote:
>
>> In addition to those great resources, what files can be checked for such
>> mechanism (only write ahead log not other zk service components),
>> particularly snapshot and log?
>> Thanks for all the input. Much appreciated!
>>
>>> -------- Original Message --------
>>> Subject: Re: Write Ahead Log
>>> Local Time: January 9, 2018 1:04 AM
>>> UTC Time: January 9, 2018 12:04 AM
>>> From: afine@apache.org
>>> To: user@zookeeper.apache.org
>>> In addition to the great papers Ibrahim mentioned I'm a big fan of the book:
ZooKeeper: Distributed Process Coordination by Flavio Junqueira,‎ Benjamin Reed for getting
a high level understanding of what's going on.
>>> Thanks,
>>> Abe
>>> On Mon, Jan 8, 2018, at 05:27, Ibrahim El-sanosi (PGR) wrote:
>>>
>>>> Hi,
>>>> You can read page 8 in [1] and section VII. EVALUATION in [2]. I hope
>>>> these will be helpful.
>>>> [1] Hunt, Patrick, et al. "ZooKeeper: Wait-free Coordination for
>>>> Internet-scale Systems." USENIX annual technical conference. Vol. 8.
>>>> 2010.
>>>> [2] Junqueira, Flavio P., Benjamin C. Reed, and Marco Serafini. "Zab:
>>>> High-performance broadcast for primary-backup systems." Dependable
>>>> Systems & Networks (DSN), 2011 IEEE/IFIP 41st International Conference
>>>> on. IEEE, 2011.
>>>> Ibrahim
>>>> -----Original Message-----
>>>> From: tj5527 [mailto:tj5527@protonmail.com]
>>>> Sent: 08 January 2018 13:14
>>>> To: user@zookeeper.apache.org
>>>> Subject: Write Ahead Log
>>>> I heard that ZooKeeper uses Write Ahead Log for preserving operation
>>>> information, and am interested in how it's implemented. But when
>>>> checking ZooKeeper confluence wiki and searching the internet, I don't
>>>> find much related documents. Only a few source files such as
>>>> FileTxnLog.java that seems to be related with some lines written
>>>> directly about write ahead log string.
>>>> https://github.com/apache/zookeeper/blob/5c4e44332e55bbf21ca59583f3e8ca97fc4bb266/src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java
>>>> Is this the right place to start? What other resources available out
>>>> there as reference?
>>>> Thanks
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message