hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Reed <br...@yahoo-inc.com>
Subject Re: Is it possible to read/write a ledger concurrently
Date Fri, 22 Oct 2010 07:41:20 GMT
  in hedwig one hub does both the publish and subscribe for a given 
topic and therefore is the only processes reading and writing from/to a 
ledger, so there isn't an issue.

The ReadAheadCache does read-ahead :) it is so that we can minimize 
latency when doing sequential reads.

ben

On 10/21/2010 11:30 PM, amit jaiswal wrote:
> Hi,
>
> How does Hedwig handles this scenario? Since only one of the hubs have the
> ownership of a topic, the same hub is able to serve both publish and subscribe
> requests concurrently. Is my understanding correct ?
>
> Also, what is the purpose of ReadAheadCache class in Hedwig? Is it used
> somewhere for this concurrent read/write problem?
>
> -regards
> Amit
>
> ----- Original Message ----
> From: Benjamin Reed<breed@yahoo-inc.com>
> To: zookeeper-user@hadoop.apache.org
> Sent: Fri, 22 October, 2010 11:09:07 AM
> Subject: Re: Is it possible to read/write a ledger concurrently
>
> currently program1 can read and write to an open ledger, but program2 must wait
> for the ledger to be closed before doing the read. the problem is that program2
> needs to know the last valid entry in the ledger. (there may be entries that may
> not yet be valid.) for performance reasons, only program1 knows the end. so you
> need a way to propagate that information.
>
> we have talked about a way to push the last entry into the bookkeeper handle.
> flavio was working on it, but i don't think it has been implemented.
>
> ben
>
> On 10/21/2010 10:22 PM, amit jaiswal wrote:
>> Hi,
>>
>> In BookKeeper documentation, the sample program creates a ledger, writes some
>> entries and then *closes* the ledger. Then a client program opens the ledger,
>> and reads the entries from it.
>>
>> Is it possible for program1 to write to a ledger, and program2 to read from
> the
>> ledger at the same time. In BookKeeper code, if a client tries to read from a
>> ledger which is not being closed (as per its metadata in zk), then a recovery
>> process is started to check for consistency.
>>
>> Waiting for ledger to get closed can introduce lot of latency at the client
>> side. Can somebody explain this functionality?
>>
>> -regards
>> Amit


Mime
View raw message