ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: Consistent ID specification from previous random UUID
Date Tue, 12 Mar 2019 16:32:43 GMT
Igniters,

I came across the same issue during development and found no sane
workaround for this issue. I believe the solution should be as simple as
possible because we are already adding a warning to let users know that it
is good to specify a consistent ID in production deployments.

As for the current solution, I do not like adding a new configuration
property like 'storageFolder' because it's another way to shoot yourself in
the leg (e.g. different nodes have different consistent IDs but configured
to have the same storage folder).
Why can't we:
 - either check if consistent ID is an instance of UUID and then take the
appropriate folder. This approach is straightforward, but may affect
current users
 - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
the consistent ID is set to UUID, and the storage folder is chosen
according to the proper rules. This change has a minimal chance to affect
current users because it's unlikely that somebody is using auto-generated
folder naming scheme as consistent ID.

Thoughts?

вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <dpavlov@apache.org>:

> Hi Igniters,
>
> A full description can be found at wiki page
>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
>
> I like the idea of introducing a property like nodeStorageName to specify
> exactly name of a directory to use. For example, it could have higher
> priority then IgniteConfiguration.getConsistentId(). It will impact the
> mentioned algorithm, but it seems to be easier to understand by users.
>
> If nobody else minds, I will try to implement this idea.
>
> Sincerely,
> Dmitriy Pavlov
>
>
> сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <vololo100@gmail.com>:
>
> > Dmitriy,
> >
> > You wrote
> > > we need an order to scan and lock random-UUID based folders.
> >
> > It would be great if you provide a discussion about that order to
> > complete the picture. Currently I cannot understand why the order is
> > important.
> >
> > Also, couple of raw thoughts:
> > 1. Can we extend a directory lookup procedure when consistent id is
> > specified to check nodeXX-consistentId directories as well?
> > 2. Can we introduce a property like nodeStorageName to specify exact
> > name of a directory to use? It looks like a straightforward and
> > universal workaround. Or is node index better in some sense? Why?
> >
> > Please, share your thoughts.
> >
> > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <dpavlov@apache.org>:
> > >
> > > Hi Ivan,
> > >
> > > Yes, you catch me, I'm a little bit cheating with lazy consensus on
> code
> > > modification without providing a PR because I was expecting that nobody
> > > comes to discussion. I will prepare PR shortly. And since we anyway
> have
> > a
> > > discussion, I will not apply anything by lazy approval.
> > >
> > > - storageNodeIndex without consistent ID will not work.
> > >  cfg.getDataStorageConfiguration().setNodeIdx() will be required only
> for
> > > case we have consistent ID.
> > >
> > > Hi Stanislav,
> > >
> > > We can't use only consistent ID because
> > >
> > > 1) we need an order to scan and lock random-UUID based folders.  Node
> > index
> > > provides the order of scan. I can find the corresponding discussion,
> but
> > I
> > > guess it is not needed.
> > > 2) we need to separate backward compatible folders from new random-UUID
> > > based folders. Using UUID as folder will not allow us to scan only new
> > name
> > > format folders.
> > >
> > > I guess specifying node index is a quite rare case and good JavaDoc
> will
> > > always help.
> > >
> > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
> > > notes:
> > > Node index used for persistent store folders in case several nodes
> reuse
> > > one persistent store root folder.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <vololo100@gmail.com>:
> > >
> > > > Dmitiy,
> > > >
> > > > Could please clarify one thing:
> > > > 1. Will it be enough to use only storageNodeIndex in order to reuse
> > > > the same persistence folders when consistentId is auto-generated?
> E.g.
> > > > I have a configuration with storageNodeIndex=1 and without explicitly
> > > > specified consistentId, will the node after restart use the same
> > > > persistence folder as before restart?
> > > >
> > > > Also a side note:
> > > > > Please share your vision. I'm going to apply this change by lazy
> > > > consensus
> > > > in 3 days.
> > > > What do you mean by "apply"? I have not seen any PR yet.
> > > >
> > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <dpavlov@apache.org>:
> > > > >
> > > > > Hi Igniters,
> > > > >
> > > > > I would like to fix the issue
> > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> specifying
> > some
> > > > > previous randomly generated UUID as a new consistent ID. Folder
> > > > generation
> > > > > algorithm here (
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > )
> > > > > allows two options
> > > > > -node00+random UUID
> > > > > - consistendId
> > > > >
> > > > > I would like to add to Ignite configuration new property nodeIndex
> in
> > > > > addition to consistent Id. New Property will be named as
> > > > storageNodeIndex,
> > > > > int, zero-based.
> > > > > This will add the third option of subfolders processing:
> > > > > node{storageNodeIndex}+consistentID
> > > > >
> > > > > Please share your vision. I'm going to apply this change by lazy
> > > > consensus
> > > > > in 3 days.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message