ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: Consistent ID specification from previous random UUID
Date Wed, 13 Mar 2019 06:43:44 GMT
Alex,

Why do we need
>  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> the consistent ID is set to UUID, and the storage folder is chosen
> according to the proper rules. This change has a minimal chance to affect
> current users because it's unlikely that somebody is using auto-generated
> folder naming scheme as consistent ID.
?
It looks hacky as well. The thing I do not like here is that a
consistentId specified in configuration is not a consistentId used by
a node sometimes.

Can we go just with
> - either check if consistent ID is an instance of UUID and then take the
> appropriate folder. This approach is straightforward, but may affect
> current users
?

And as a last chance a user will have a possibility to rename a directory.


вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk <alexey.goncharuk@gmail.com>:
>
> Igniters,
>
> I came across the same issue during development and found no sane
> workaround for this issue. I believe the solution should be as simple as
> possible because we are already adding a warning to let users know that it
> is good to specify a consistent ID in production deployments.
>
> As for the current solution, I do not like adding a new configuration
> property like 'storageFolder' because it's another way to shoot yourself in
> the leg (e.g. different nodes have different consistent IDs but configured
> to have the same storage folder).
> Why can't we:
>  - either check if consistent ID is an instance of UUID and then take the
> appropriate folder. This approach is straightforward, but may affect
> current users
>  - check if consistent ID is set to a string 'nodeXX-UUID'. In this case
> the consistent ID is set to UUID, and the storage folder is chosen
> according to the proper rules. This change has a minimal chance to affect
> current users because it's unlikely that somebody is using auto-generated
> folder naming scheme as consistent ID.
>
> Thoughts?
>
> вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <dpavlov@apache.org>:
>
> > Hi Igniters,
> >
> > A full description can be found at wiki page
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration
> >
> > I like the idea of introducing a property like nodeStorageName to specify
> > exactly name of a directory to use. For example, it could have higher
> > priority then IgniteConfiguration.getConsistentId(). It will impact the
> > mentioned algorithm, but it seems to be easier to understand by users.
> >
> > If nobody else minds, I will try to implement this idea.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> >
> > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <vololo100@gmail.com>:
> >
> > > Dmitriy,
> > >
> > > You wrote
> > > > we need an order to scan and lock random-UUID based folders.
> > >
> > > It would be great if you provide a discussion about that order to
> > > complete the picture. Currently I cannot understand why the order is
> > > important.
> > >
> > > Also, couple of raw thoughts:
> > > 1. Can we extend a directory lookup procedure when consistent id is
> > > specified to check nodeXX-consistentId directories as well?
> > > 2. Can we introduce a property like nodeStorageName to specify exact
> > > name of a directory to use? It looks like a straightforward and
> > > universal workaround. Or is node index better in some sense? Why?
> > >
> > > Please, share your thoughts.
> > >
> > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov <dpavlov@apache.org>:
> > > >
> > > > Hi Ivan,
> > > >
> > > > Yes, you catch me, I'm a little bit cheating with lazy consensus on
> > code
> > > > modification without providing a PR because I was expecting that nobody
> > > > comes to discussion. I will prepare PR shortly. And since we anyway
> > have
> > > a
> > > > discussion, I will not apply anything by lazy approval.
> > > >
> > > > - storageNodeIndex without consistent ID will not work.
> > > >  cfg.getDataStorageConfiguration().setNodeIdx() will be required only
> > for
> > > > case we have consistent ID.
> > > >
> > > > Hi Stanislav,
> > > >
> > > > We can't use only consistent ID because
> > > >
> > > > 1) we need an order to scan and lock random-UUID based folders.  Node
> > > index
> > > > provides the order of scan. I can find the corresponding discussion,
> > but
> > > I
> > > > guess it is not needed.
> > > > 2) we need to separate backward compatible folders from new random-UUID
> > > > based folders. Using UUID as folder will not allow us to scan only new
> > > name
> > > > format folders.
> > > >
> > > > I guess specifying node index is a quite rare case and good JavaDoc
> > will
> > > > always help.
> > > >
> > > > DataStorageConfiguration().setNodeIdx()  JavaDoc may include following
> > > > notes:
> > > > Node index used for persistent store folders in case several nodes
> > reuse
> > > > one persistent store root folder.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван <vololo100@gmail.com>:
> > > >
> > > > > Dmitiy,
> > > > >
> > > > > Could please clarify one thing:
> > > > > 1. Will it be enough to use only storageNodeIndex in order to reuse
> > > > > the same persistence folders when consistentId is auto-generated?
> > E.g.
> > > > > I have a configuration with storageNodeIndex=1 and without explicitly
> > > > > specified consistentId, will the node after restart use the same
> > > > > persistence folder as before restart?
> > > > >
> > > > > Also a side note:
> > > > > > Please share your vision. I'm going to apply this change by
lazy
> > > > > consensus
> > > > > in 3 days.
> > > > > What do you mean by "apply"? I have not seen any PR yet.
> > > > >
> > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov <dpavlov@apache.org>:
> > > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > I would like to fix the issue
> > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 about
> > specifying
> > > some
> > > > > > previous randomly generated UUID as a new consistent ID. Folder
> > > > > generation
> > > > > > algorithm here (
> > > > > >
> > > > >
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
> > > > > )
> > > > > > allows two options
> > > > > > -node00+random UUID
> > > > > > - consistendId
> > > > > >
> > > > > > I would like to add to Ignite configuration new property nodeIndex
> > in
> > > > > > addition to consistent Id. New Property will be named as
> > > > > storageNodeIndex,
> > > > > > int, zero-based.
> > > > > > This will add the third option of subfolders processing:
> > > > > > node{storageNodeIndex}+consistentID
> > > > > >
> > > > > > Please share your vision. I'm going to apply this change by
lazy
> > > > > consensus
> > > > > > in 3 days.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message