ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Pavlov <dpavlov....@gmail.com>
Subject Re: Ignite Durable Memory & Persistent Store introduction for developers
Date Fri, 30 Mar 2018 16:52:02 GMT
Hi Ivan,

Excellent addition to the article. Thank you very much for contribution.

I hope Igniters, especially Native Persistence experts will follow your
lead.

Sincerely,
Dmitriy Pavlov

пт, 30 мар. 2018 г. в 11:38, Ivan Rakov <ivan.glukos@gmail.com>:

> Igniters,
>
> I've added some info about Pages Write Throttling and WAL compaction to
> the "Ignite Persistent Store - under the hood" document.
> Please let me know if anything is not clear enough.
>
> Best Regards,
> Ivan Rakov
>
> On 26.03.2018 0:18, Dmitry Pavlov wrote:
> > Hi Igniters,
> >
> > I've done several updates in section
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Regionandsegmentstructure
> >
> > These updates releated to optimizations of LoadedPageTable in Ignite 2.5
> > (Replacement of storage algorithm corresponding to the ID in the page,
> and
> > the location of a physical address within the segment algorithm is used
> > Robin Hood with backward shift. Previous implementation was not resistant
> > to the ‘page rotation’, a lot of the cells were flooded with Removed
> state,
> > and new algorithm does not have ‘removed’ state of cell.)
> >
> > New section has now picture of segments structure, and describes locking
> > optimisations in 2.5.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 19 окт. 2017 г. в 22:33, Dmitry Pavlov <dpavlov.spb@gmail.com>:
> >
> >> Hi Denis,
> >>
> >> I've updated the section in relation to your questions, please see
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory
> >>
> >> I’ve also moved paragraph related to page offset calculation and usage
> in
> >> files into second article (PDS)
> >>
> >> Sincerely,
> >> Dmitriy Pavlov
> >>
> >> пт, 1 сент. 2017 г. в 19:50, Denis Magda <dmagda@apache.org>:
> >>
> >>> Dmitriy,
> >>
> >>> First of all, thanks for putting this low level details on paper!
> Current
> >>> and future community members will be grateful to you.
> >>>
> >>> Second, let me be a reviewer of the page and suggest some improvements.
> >>> Until the review is over I would encourage us to hold on with any
> public
> >>> announcements such as blog post.
> >>>
> >>> I’ve reviewed and edited Ignite Durable Memory section (
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory
> >>> <
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory
> >).
> >>> Please address the questions below and I’ll move on with the new
> sections
> >>> afterwards.
> >>>
> >>> 1) "Different caches and its partitions bring more complex addressing
> >>> scheme but it is still possible to map from page ID to position in
> >>> particular file: pageId => file + offset in this file”.
> >>>
> >>> Probably this appears too early in the doc because above we just
> >>> introduced page ID notion with durable memory and now we’re talking
> about
> >>> some files? Are these files of ignite persistence? What if the
> persistence
> >>> is disabled? Honestly, the sentence gets out of the story context.
> >>>
> >>>
> >>> 2) "Also, the tree pages can be merged into one if < 50% of space is
> >>> used.”
> >>>
> >>> 50% of the whole memory available? Please clarify.
> >>>
> >>>
> >>> 3) "There is one page ID table per memory segment.”
> >>>
> >>> It’s the same segment as it’s described in the main doc, right?
> >>>
> https://apacheignite.readme.io/docs/memory-architecture#section-memory-segment
> >>> <
> >>>
> https://apacheignite.readme.io/docs/memory-architecture#section-memory-segment
> >>>
> >>> 4) "within unsafe segment”
> >>>
> >>> If we want to mention “usafe” here we need to explain what that is.
> >>>
> >>> 5) "If memory amount less than whole index size, Ignite still can
> >>> operate.”
> >>>
> >>> How do we define “index size”? Is this the page size?
> >>>
> >>>
> >>> —
> >>> Denis
> >>>
> >>>
> >>>> On Aug 30, 2017, at 8:24 AM, Dmitry Pavlov <dpavlov.spb@gmail.com>
> >>> wrote:
> >>>> Hi Igniters,
> >>>>
> >>>>
> >>>> I am glad to introduce two new wiki articles. These articles contain
> >>>> high-level description of
> >>>>
> >>>> - Ignite Durable Memory (
> >>>>
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood
> >>>> )
> >>>>
> >>>> - and Ignite Persistent Store (
> >>>>
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> >>>> )
> >>>>
> >>>> These articles cover basic principles and internal design. Pages may
> be
> >>>> useful for new contributors during start of working with persistent
> >>> store
> >>>> code base.
> >>>>
> >>>>
> >>>> I would like to thank all contributors who helped me with my
> questions.
> >>>> Their answers and support made it possible to start this description.
> >>>>
> >>>>
> >>>> I will appreciate any feedback and questions. Suggestions and edits
> are
> >>>> welcome, of course.
> >>>>
> >>>>
> >>>> Sincerely,
> >>>>
> >>>> Dmitriy Pavlov
> >>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message