ariatosca-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DeWayne Filppi <dewa...@cloudify.co>
Subject Re: Unique identification of an instance element across services
Date Fri, 15 Sep 2017 18:38:54 GMT
I was also considering a "partial hash" concept like git.

On Fri, Sep 15, 2017 at 11:04 AM, Thomas Nadeau <tnadeau@lucidvision.com>
wrote:

>
> > On Sep 15, 2017, at 1:53 PM, Tal Liron <tal@cloudify.co> wrote:
> >
> > When do you actually have to ender node names? Probably only in "aria
> nodes
> > show". And in those cases you would be copy-pasting from a list. We could
> > also improve our CLI completion code to properly complete node IDs.
>
>         That sounds like a very useful enhancement.  Do we have a Jira for
> this yet? *)
>
> > I think the serial numbers are more confusing than helpful. Let's say you
> > currently have 20 difference services running, and they are of various
> > different service templates. But let's say a few service templates have
> > node templates with the same name, "database". You could potentially
> > "database_1" in the list and "database_2", but each one of these nodes
> > would be of a different node template of a different service template.
> The
> > serial number gives the false sense that these two nodes are somehow
> > together. Anyway, we discussed this in much detail already: we all agree
> > that the serial system is totally broken if you're using more than ARIA
> > install, or even if a few different ARIA users are using the same cloud
> > accounts (each ARIA install could create its own "database_1" -- what if
> > you have two hosts with that same DNS name?).
>
>         I was just going to say the point you made above about DNS name
> overlap.
> It sounds like we need to sit down and re-visit the serial number
> management?
>
>         —Tom
>
>
> >
> >
> > On Fri, Sep 15, 2017 at 12:35 PM, DeWayne Filppi <dewayne@cloudify.co>
> > wrote:
> >
> >> I get the feeling that you are more gifted typist than most.  Or are you
> >> saying nobody will ever be required to type in one those IDs?
> >>
> >> On Fri, Sep 15, 2017 at 9:27 AM, Tal Liron <tal@cloudify.co> wrote:
> >>
> >>> Before we allow users to configure this, we have another JIRA to
> resolve:
> >>> actually, we don't have a mechanism for storing configuration yet! Here
> >> is
> >>> the open JIRA:
> >>>
> >>> https://issues.apache.org/jira/browse/ARIA-229
> >>>
> >>> As for what to configure in this case, our practice until now was that
> >> the
> >>> UUID would be added as an underscored postfix of the object's name. So
> if
> >>> you have a node template named "database" then node instances could be,
> >>> assuming longest form of UUID (alphanumeric, 36 characters):
> >>>
> >>> database_2d79fd86-0877-49ca-81d8-cd2dc9f7b0e2
> >>> database_2819972e-3b07-4923-be94-43e95985155f
> >>> database_45b9faf5-8bf4-482a-a570-d1c058270424
> >>>
> >>> This guarantees names that are universally unique and yet still
> >> meaningful:
> >>> you would be able to tell at a glance what kind of node this is: a
> >>> database. Note that we also have a mechanism in place to warn you if
> the
> >>> final name is more than 63 characters, because such names can't be used
> >> as
> >>> DNS hostnames (a common usage for node names in the cloud). This should
> >>> also be configurable.
> >>>
> >>> I don't see why this needs to be abstracted from the user. If you are
> >> using
> >>> the CLI and see the list of nodes, you can refer to the node you are
> >>> examining with the full name as seen above. I think having a separate
> UI
> >>> name such as "database_1", "database_2', etc., would be confusing.
> >>>
> >>> So, assuming the above, I imagine these kinds of configuration vars:
> >>>
> >>> instance.id_type = 'uuid' | 'alphanumeric' | 'base57' (default?) |
> >> 'serial'
> >>> node.id_max_length = 63
> >>>
> >>> Here are examples of the other types. Alphanumeric (25 characters):
> >>>
> >>> database_t5evps77wp5biqdb1oyw36956
> >>> database_uw5oa530kn9mu73lzjuech02a
> >>> database_nzv3a7umph0g1093abwq6qjd3
> >>>
> >>> And base57 (22 characters):
> >>>
> >>> database_g8KV4qpKep2J2uA473fv6X
> >>> database_M2bLkYsToZ52L3HSy7CCmC
> >>> database_q8se9o5fDDWvT53tnnRiXN
> >>>
> >>> My personal preference for the default has always been base57. It is
> both
> >>> the most compact, meaning less of a chance you would hit the 63
> character
> >>> limit, and also cleverly designed for human readability (no
> >>> visually-ambiguous glyphs).
> >>>
> >>>
> >>>
> >>> On Fri, Sep 15, 2017 at 2:34 AM, Vaishnavi K.R <
> >> vaishnavi.k.r@ericsson.com
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>>
> >>>> Thanks for the update.
> >>>>
> >>>>
> >>>> With current ARIA, the utility module to generate the UUID is
> >> available.
> >>>> But the UUID support will also mandate the following changes if my
> >>>> understanding is right,
> >>>>
> >>>>  1.  the inclusion of the UUID column in the mapper classes of
> >>> sqlalchemy
> >>>>  2.  the model object created should set the value for the UUID and
> >> send
> >>>> it to database
> >>>>
> >>>> For an use case in our product, we badly need this UUID based element
> >>>> identification. So I look forward to your comments on the following,
> >>>>
> >>>>
> >>>>  1.  We contribute the UUID support to ARIA without affecting the
> >>> current
> >>>> CLI module i.e. CLI will continue to use the name option. The UUID
> will
> >>> be
> >>>> completely abstracted from the user.
> >>>>  2.  Configurable option to use UUID or name based identification. By
> >>>> default, it will work with the name based identification
> >>>>
> >>>> Also I need clarification on the UUID generation. Currently ARIA
> >> supports
> >>>> four variants. Do we have any standard on how this UUID should be and
> >>> also
> >>>> on what aspect these four variants are concluded on?
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>> /Vaish
> >>>>
> >>>> ________________________________
> >>>> From: Tal Liron <tal@cloudify.co>
> >>>> Sent: Tuesday, September 5, 2017 8:24:41 PM
> >>>> To: dev@ariatosca.incubator.apache.org
> >>>> Subject: Re: Unique identification of an instance element across
> >> services
> >>>>
> >>>> We already have utility code to generate all kinds of UUIDs, so it's
> >>>> trivial to make the change. I guess it's just a matter of making a
> >>>> decision...
> >>>>
> >>>> On Tue, Sep 5, 2017 at 3:57 AM, Vaishnavi K.R <
> >>> vaishnavi.k.r@ericsson.com>
> >>>> wrote:
> >>>>
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I agree that with the CLI based usage in ARIA, the requirement of
the
> >>>> UUID
> >>>>> based identification of the node and service instance elements is
an
> >>>>> overhead.
> >>>>>
> >>>>> From the discussions so far, it seems like UUID is important in
> >>> handling
> >>>>> the multi-user and multi-tenant scenarios.
> >>>>>
> >>>>> Do you have any update on when UUID will be considered in the
> >> roadmap?
> >>>>> If its not too far, we would like to make our contribution to ARIA
on
> >>>> UUID.
> >>>>>
> >>>>> Looking forward to your response.
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> /Vaish
> >>>>>
> >>>>> ________________________________
> >>>>> From: Avia Efrat <avia@cloudify.co>
> >>>>> Sent: Sunday, July 30, 2017 10:35:45 PM
> >>>>> To: dev@ariatosca.incubator.apache.org
> >>>>> Subject: Re: Unique identification of an instance element across
> >>> services
> >>>>>
> >>>>> First, good arguments from both 'sides'.
> >>>>>
> >>>>> I am for at least adding a uuid as an option, as ARIA is intended
to
> >> be
> >>>>> used at scale as well.
> >>>>> But currently, I am for the simple ids to be used as default (and
not
> >>>>> uuids). Like it or not, right now ARIA is more at a 'TOSCA
> >> playground'
> >>>>> stage, and I think that's perfectly fine =)
> >>>>> And at this stage, I think simple ids will be better, as they easier
> >> to
> >>>> use
> >>>>> via the CLI, but more importantly, don't clog the logs with long
> >>>>> meaningless strings. As ARIA matures, we could switch the default
to
> >>>> UUIDs.
> >>>>>
> >>>>> And BTW, as our log format is configurable, there could be other
ways
> >>>> than
> >>>>> UUIDs to distinguish between nodes with the 'same id' in a central
> >>>> logging
> >>>>> system, e.g using the user name as another indicator.
> >>>>>
> >>>>> On Thu, Jul 27, 2017 at 10:20 AM, Vaishnavi K.R <
> >>>>> vaishnavi.k.r@ericsson.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>>
> >>>>>> Thanks for the active discussion.
> >>>>>>
> >>>>>>
> >>>>>> Having UUID at the node instance level will just make the nodes
> >>> unique.
> >>>>>>
> >>>>>> And these names will not be used by the user directly as no
> >>> operations
> >>>>> are
> >>>>>> happening on the node instance name.
> >>>>>>
> >>>>>> But at the service template and the service level, UUID will
be of
> >>>> great
> >>>>>> help considering the multi user and multi tenancy situations.
> >>>>>>
> >>>>>>
> >>>>>> So in a large scale perspective, the node names and the service
> >>>> template
> >>>>>> names have high probability of being same.
> >>>>>>
> >>>>>> When these enter into the automated world, it will create more
> >>> problem
> >>>>>> when name conflicts occur and its adds overhead to make changes
> >> every
> >>>>> time
> >>>>>> to overcome the conflict.
> >>>>>>
> >>>>>>
> >>>>>> UUID at service template and the service level: will be of much
use
> >>> in
> >>>>> the
> >>>>>> above scenario and operations by user on these are less
> >>>>>>
> >>>>>> UUID at node instance level: makes the node much unique and
no
> >>>> operation
> >>>>>> happens on it
> >>>>>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> /Vaish
> >>>>>>
> >>>>>> ________________________________
> >>>>>> From: Tal Liron <tal@cloudify.co>
> >>>>>> Sent: Wednesday, July 26, 2017 8:48:40 PM
> >>>>>> To: dev@ariatosca.incubator.apache.org
> >>>>>> Subject: Re: Unique identification of an instance element across
> >>>> services
> >>>>>>
> >>>>>> I just don't see users having to deal much with node IDs outside
of
> >>>>> simple
> >>>>>> hello-world style tutorials, and I'd hate for the first impressions
> >>>> that
> >>>>>> users get out of ARIA is that it's just a playground for TOSCA.
It
> >>>> should
> >>>>>> be ready out-of-the-box for the real world.
> >>>>>>
> >>>>>> On Wed, Jul 26, 2017 at 9:13 AM, DeWayne Filppi <
> >> dewayne@cloudify.co
> >>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Such is their strength.  I'm just advocating using them
as a last
> >>>>> resort
> >>>>>>> because they are user unfriendly and gigantic.
> >>>>>>>
> >>>>>>> On Tue, Jul 25, 2017 at 12:55 PM, Tal Liron <tal@cloudify.co>
> >>> wrote:
> >>>>>>>
> >>>>>>>> Let's consider a mass deployment: thousands of service
> >> instances
> >>> of
> >>>>> the
> >>>>>>>> same service template, created by many different users
with
> >> their
> >>>> own
> >>>>>>> ARIA
> >>>>>>>> installations (and databases). In that case, assuming
we use
> >>>>> sequential
> >>>>>>>> IDs, you would have the same node ID appear many times.
You
> >> would
> >>>>> have
> >>>>>> to
> >>>>>>>> identify it via the particular user and service instance.
If
> >>> you're
> >>>>>>>> centralizing logs, this can quickly be cumbersome. A
UUID will
> >>>>> identify
> >>>>>>> it
> >>>>>>>> globally and avoid any confusion.
> >>>>>>>>
> >>>>>>>> I think the default should be something that avoids
such
> >>> problems.
> >>>>> For
> >>>>>>>> users who insist on shorter IDs, we can allow them to
configure
> >>> it.
> >>>>>>>>
> >>>>>>>> On Tue, Jul 25, 2017 at 2:42 PM, DeWayne Filppi <
> >>>> dewayne@cloudify.co
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> True uuids are seductive, because of their simplicity.
 But
> >>> they
> >>>>> are
> >>>>>>>> huge,
> >>>>>>>>> overkill, and meaningless.  Imho a structured id
is superior
> >> if
> >>>> it
> >>>>>> can
> >>>>>>> be
> >>>>>>>>> made to work without a global locking scheme.
> >>>>>>>>>
> >>>>>>>>> - DeWayne
> >>>>>>>>>
> >>>>>>>>> On Jul 25, 2017 12:11 PM, "Tal Liron" <tal@cloudify.co>
> >> wrote:
> >>>>>>>>>
> >>>>>>>>>> It's not an issue of thread safety -- it could
be entirely
> >>>>>> different
> >>>>>>>>>> processes, on different machines, accessing
the same db. It
> >>> can
> >>>>> be
> >>>>>>>> solved
> >>>>>>>>>> via a SQL transaction, but I feel the whole
issue can be
> >>>> avoided
> >>>>> by
> >>>>>>>> using
> >>>>>>>>>> UUIDs.
> >>>>>>>>>>
> >>>>>>>>>> Using the CLI to access specific nodes is not
something I
> >> see
> >>>>>>>> happening a
> >>>>>>>>>> lot outside of debugging. And when you do debug,
you'll
> >>>> probably
> >>>>> be
> >>>>>>>>> copying
> >>>>>>>>>> and pasting a node ID from the logs, so shorter
names do
> >> not
> >>>> add
> >>>>>> much
> >>>>>>>>> ease
> >>>>>>>>>> of use.
> >>>>>>>>>>
> >>>>>>>>>> Again, I would be personally happiest if this
was
> >>> configurable
> >>>>> (and
> >>>>>>>>>> personally think UUIDs should be the reasonable
default).
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jul 25, 2017 at 2:01 PM, Maxim Orlov
<
> >>>> maxim@cloudify.co>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Technically we have no issue with implementing
this via
> >>> uuid
> >>>>> or a
> >>>>>>>>>>> threadsafe solution for the current index
implementation.
> >>>>>>>>>>>
> >>>>>>>>>>> Getting node data via the cli feels more
intuitive using
> >>> the
> >>>>>> index
> >>>>>>>>> based
> >>>>>>>>>>> ID, rather than the uuid based ID in my
opionion.
> >>>>>>>>>>>
> >>>>>>>>>>> On Jul 25, 2017 9:49 PM, "Tal Liron" <tal@cloudify.co>
> >>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Our code for determining the next index
is not
> >> concurrently
> >>>>> safe
> >>>>>>> (no
> >>>>>>>>>> atomic
> >>>>>>>>>>> transaction) so I can see it breaking in
concurrent use
> >>> cases
> >>>>>>>> (running
> >>>>>>>>>> two
> >>>>>>>>>>> ARIA commands at the same time).
> >>>>>>>>>>>
> >>>>>>>>>>> What is to gain here in terms of human readability?
In my
> >>>>> opinion
> >>>>>>> it
> >>>>>>>>> adds
> >>>>>>>>>>> confusion because it gives a false sense
of
> >> predictability.
> >>>>>>>>>>>
> >>>>>>>>>>> In my opinion the best compromise is to
use
> >> base57-encoded
> >>>>> UUIDs.
> >>>>>>>> These
> >>>>>>>>>> are
> >>>>>>>>>>> true UUIDs, but use a mix of upper and lowercase
> >>>> alphanumerics
> >>>>>>>> ensuring
> >>>>>>>>>> no
> >>>>>>>>>>> visually ambiguous characters. We have the
code for this
> >> in
> >>>>>>>>>> utils/uuid.py.
> >>>>>>>>>>>
> >>>>>>>>>>> See also: https://github.com/wyattisimo/base57-ruby
> >>>>> [https://avatars1.githubusercontent.com/u/625546?v=3&s=400]<https://
> >>>>> github.com/wyattisimo/base57-ruby>
> >>>>>
> >>>>> GitHub - wyattisimo/base57-ruby: Base57 encoder for Ruby.<
> >>>>> https://github.com/wyattisimo/base57-ruby>
> >>>>> github.com
> >>>>> base57-ruby - Base57 encoder for Ruby. ... Clone with HTTPS Use
Git
> >> or
> >>>>> checkout with SVN using the web URL.
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jul 25, 2017 at 1:28 PM, Maxim Orlov
<
> >>>>> maxim@cloudify.co>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Actually the refactoring was made so
the id would be
> >> more
> >>>>> user
> >>>>>>>>>> readable.
> >>>>>>>>>>>> The index is determined according to
the used indices
> >>> (it's
> >>>>> not
> >>>>>>>> just
> >>>>>>>>> a
> >>>>>>>>>>>> running number). If indeed this poses
an issue (or if
> >>>> indeed
> >>>>> a
> >>>>>>> uuid
> >>>>>>>>> is
> >>>>>>>>>>>> easier to recognize, or even use in
a query), let's
> >>> discuss
> >>>>> it
> >>>>>>>>>> further...
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jul 25, 2017 at 7:35 PM, Tal
Liron <
> >>>> tal@cloudify.co>
> >>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> We used to use UUIDs but at some
point this was
> >>>>> refactored. I
> >>>>>>>> tend
> >>>>>>>>> to
> >>>>>>>>>>>> agree
> >>>>>>>>>>>>> with you.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Actually, I would prefer it to be
configurable. We
> >> have
> >>>>> code
> >>>>>> in
> >>>>>>>>> place
> >>>>>>>>>>> for
> >>>>>>>>>>>>> ID generation of various types:
UUIDs, short UUIDs,
> >> and
> >>>>>>>>> sequentials.
> >>>>>>>>>>> All
> >>>>>>>>>>>> of
> >>>>>>>>>>>>> them would seem useful to me for
various scenarios.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jul 25, 2017 at 3:42 AM,
Vaishnavi K.R <
> >>>>>>>>>>>> vaishnavi.k.r@ericsson.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> With my understanding in current
ARIA, the node
> >>>> instances
> >>>>>> are
> >>>>>>>>> made
> >>>>>>>>>>>> unique
> >>>>>>>>>>>>>> by prefixing the node name with
the 'id of the
> >>> service'
> >>>>>> (i.e.
> >>>>>>>> the
> >>>>>>>>>>>> primary
> >>>>>>>>>>>>>> key of the service table) as
the instances are
> >>> specific
> >>>>> to
> >>>>>>> the
> >>>>>>>>>>> service.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What will be the name of the
node instances if the
> >>>>> default
> >>>>>>>>>> instances
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the node template is '3' and
how this will hold
> >> good
> >>>>> during
> >>>>>>>> scale
> >>>>>>>>>> in
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>> out?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Could UUID be of great help
in handling such cases
> >> by
> >>>>>>> including
> >>>>>>>>>> that
> >>>>>>>>>>>> as a
> >>>>>>>>>>>>>> column in the database tables
of the service and
> >> the
> >>>>> node?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This will wipe out the naming
confusions and
> >> querying
> >>>> can
> >>>>>> be
> >>>>>>>> made
> >>>>>>>>>>> easy
> >>>>>>>>>>>>>> with the UUIDs.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Looking forward to your suggestion.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> /Vaish
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message