asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: Force LSM component flush & NC-CC messaging ACK
Date Sat, 21 Jan 2017 16:17:33 GMT
I believe Ildar is just looking for a way to ensure, in doing 
experiments, that things are all in disk components.  His 
stats-gathering extensions camp on the LSM lifecycle - flushes in 
particular - and he wants to finish that process in his testing and 
experiments.  Wail's schema inference stuff has a similar flavor.  So 
the goal is to flush any lingering memory components to disk for a given 
dataset at the end of the "experiment lifecycle".

We have DDL to compact a dataset - which flushes AND compacts - it might 
also be useful to have DDL to flush a dataset without also forcing 
compaction - as a way for an administrator to release that dataset's 
in-memory component related resources.  (Not that it's "necessary" for 
any correctness reason - just might be nice to be able to do that.  That 
could also be useful in scripting more user-level-oriented recovery tests.)

Thus, I'd likely vote for adding a harmless new DDL statement - another 
arm of the one that supports compaction - for this.

Cheers,

Mike


On 1/21/17 6:21 AM, Till Westmann wrote:
> Hi Ildar,
>
> On 19 Jan 2017, at 4:02, Ildar Absalyamov wrote:
>
>> Since I was out for quite a while and a lot of things happened in a 
>> meantime in a codebase I wanted to clarify couple of things.
>>
>> I was wondering if there is any legitimate way to force the data of 
>> in-memory components to be flushed, other then stop the whole instance?
>> It used to be that choosing a different default dataverse with “use” 
>> statement did that trick, but that is not the case anymore.
>
> Just wondering, why do you want to flush the in-memory components to 
> disk?
>
>> Another question is regarding CC<->NC & NC<->NC messaging. Does the

>> sender get some kind of ACK that the message was received by the 
>> addressee? Say if I send a message just before the instance shutdown 
>> will the shutdown hook wait until the message is delivered and 
>> processed?
>
> I agree with Murtadha, that I can certainly be done. However, we also 
> need to assume that some shutdowns won’t be clean and so the messages 
> might not be received. So it might be easier to just be able to 
> recover from missing messages than to be able to recover *and* to 
> synchronize on shutdown. Just a thought - maybe that’s not even an 
> issue for your use-case.
>
> Cheers,
> Till


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message