ariatosca-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ran Ziv <...@cloudify.co>
Subject Re: Unique identification of an instance element across services
Date Fri, 15 Sep 2017 18:42:22 GMT
I can't seem to be able to access our JIRA at the moment, but generally
speaking, the CLI currently supports "static completion" only, i.e. it
auto-completes CLI commands but not object names.
We tried implementing dynamic completion (e.g. tab on "-s" would
auto-complete service names from the storage), but we ran into some issues
with the underlying Click framework.
I'm not sure if an issue for trying to implement this further is currently
open on our JIRA.

Regarding a "partial hash" concept, I don't really find this to be useful
in this case. IMO, as Tal's mentioned, the cases when you need to actually
use these auto-generated long names are rare, and when that happens,
dynamic completion can take care of it well, if we can get it done.



On Fri, Sep 15, 2017 at 9:04 PM, Thomas Nadeau <tnadeau@lucidvision.com>
wrote:

>
> > On Sep 15, 2017, at 1:53 PM, Tal Liron <tal@cloudify.co> wrote:
> >
> > When do you actually have to ender node names? Probably only in "aria
> nodes
> > show". And in those cases you would be copy-pasting from a list. We could
> > also improve our CLI completion code to properly complete node IDs.
>
>         That sounds like a very useful enhancement.  Do we have a Jira for
> this yet? *)
>
> > I think the serial numbers are more confusing than helpful. Let's say you
> > currently have 20 difference services running, and they are of various
> > different service templates. But let's say a few service templates have
> > node templates with the same name, "database". You could potentially
> > "database_1" in the list and "database_2", but each one of these nodes
> > would be of a different node template of a different service template.
> The
> > serial number gives the false sense that these two nodes are somehow
> > together. Anyway, we discussed this in much detail already: we all agree
> > that the serial system is totally broken if you're using more than ARIA
> > install, or even if a few different ARIA users are using the same cloud
> > accounts (each ARIA install could create its own "database_1" -- what if
> > you have two hosts with that same DNS name?).
>
>         I was just going to say the point you made above about DNS name
> overlap.
> It sounds like we need to sit down and re-visit the serial number
> management?
>
>         —Tom
>
>
> >
> >
> > On Fri, Sep 15, 2017 at 12:35 PM, DeWayne Filppi <dewayne@cloudify.co>
> > wrote:
> >
> >> I get the feeling that you are more gifted typist than most.  Or are you
> >> saying nobody will ever be required to type in one those IDs?
> >>
> >> On Fri, Sep 15, 2017 at 9:27 AM, Tal Liron <tal@cloudify.co> wrote:
> >>
> >>> Before we allow users to configure this, we have another JIRA to
> resolve:
> >>> actually, we don't have a mechanism for storing configuration yet! Here
> >> is
> >>> the open JIRA:
> >>>
> >>> https://issues.apache.org/jira/browse/ARIA-229
> >>>
> >>> As for what to configure in this case, our practice until now was that
> >> the
> >>> UUID would be added as an underscored postfix of the object's name. So
> if
> >>> you have a node template named "database" then node instances could be,
> >>> assuming longest form of UUID (alphanumeric, 36 characters):
> >>>
> >>> database_2d79fd86-0877-49ca-81d8-cd2dc9f7b0e2
> >>> database_2819972e-3b07-4923-be94-43e95985155f
> >>> database_45b9faf5-8bf4-482a-a570-d1c058270424
> >>>
> >>> This guarantees names that are universally unique and yet still
> >> meaningful:
> >>> you would be able to tell at a glance what kind of node this is: a
> >>> database. Note that we also have a mechanism in place to warn you if
> the
> >>> final name is more than 63 characters, because such names can't be used
> >> as
> >>> DNS hostnames (a common usage for node names in the cloud). This should
> >>> also be configurable.
> >>>
> >>> I don't see why this needs to be abstracted from the user. If you are
> >> using
> >>> the CLI and see the list of nodes, you can refer to the node you are
> >>> examining with the full name as seen above. I think having a separate
> UI
> >>> name such as "database_1", "database_2', etc., would be confusing.
> >>>
> >>> So, assuming the above, I imagine these kinds of configuration vars:
> >>>
> >>> instance.id_type = 'uuid' | 'alphanumeric' | 'base57' (default?) |
> >> 'serial'
> >>> node.id_max_length = 63
> >>>
> >>> Here are examples of the other types. Alphanumeric (25 characters):
> >>>
> >>> database_t5evps77wp5biqdb1oyw36956
> >>> database_uw5oa530kn9mu73lzjuech02a
> >>> database_nzv3a7umph0g1093abwq6qjd3
> >>>
> >>> And base57 (22 characters):
> >>>
> >>> database_g8KV4qpKep2J2uA473fv6X
> >>> database_M2bLkYsToZ52L3HSy7CCmC
> >>> database_q8se9o5fDDWvT53tnnRiXN
> >>>
> >>> My personal preference for the default has always been base57. It is
> both
> >>> the most compact, meaning less of a chance you would hit the 63
> character
> >>> limit, and also cleverly designed for human readability (no
> >>> visually-ambiguous glyphs).
> >>>
> >>>
> >>>
> >>> On Fri, Sep 15, 2017 at 2:34 AM, Vaishnavi K.R <
> >> vaishnavi.k.r@ericsson.com
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>>
> >>>> Thanks for the update.
> >>>>
> >>>>
> >>>> With current ARIA, the utility module to generate the UUID is
> >> available.
> >>>> But the UUID support will also mandate the following changes if my
> >>>> understanding is right,
> >>>>
> >>>>  1.  the inclusion of the UUID column in the mapper classes of
> >>> sqlalchemy
> >>>>  2.  the model object created should set the value for the UUID and
> >> send
> >>>> it to database
> >>>>
> >>>> For an use case in our product, we badly need this UUID based element
> >>>> identification. So I look forward to your comments on the following,
> >>>>
> >>>>
> >>>>  1.  We contribute the UUID support to ARIA without affecting the
> >>> current
> >>>> CLI module i.e. CLI will continue to use the name option. The UUID
> will
> >>> be
> >>>> completely abstracted from the user.
> >>>>  2.  Configurable option to use UUID or name based identification. By
> >>>> default, it will work with the name based identification
> >>>>
> >>>> Also I need clarification on the UUID generation. Currently ARIA
> >> supports
> >>>> four variants. Do we have any standard on how this UUID should be and
> >>> also
> >>>> on what aspect these four variants are concluded on?
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>> /Vaish
> >>>>
> >>>> ________________________________
> >>>> From: Tal Liron <tal@cloudify.co>
> >>>> Sent: Tuesday, September 5, 2017 8:24:41 PM
> >>>> To: dev@ariatosca.incubator.apache.org
> >>>> Subject: Re: Unique identification of an instance element across
> >> services
> >>>>
> >>>> We already have utility code to generate all kinds of UUIDs, so it's
> >>>> trivial to make the change. I guess it's just a matter of making a
> >>>> decision...
> >>>>
> >>>> On Tue, Sep 5, 2017 at 3:57 AM, Vaishnavi K.R <
> >>> vaishnavi.k.r@ericsson.com>
> >>>> wrote:
> >>>>
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I agree that with the CLI based usage in ARIA, the requirement of
the
> >>>> UUID
> >>>>> based identification of the node and service instance elements is
an
> >>>>> overhead.
> >>>>>
> >>>>> From the discussions so far, it seems like UUID is important in
> >>> handling
> >>>>> the multi-user and multi-tenant scenarios.
> >>>>>
> >>>>> Do you have any update on when UUID will be considered in the
> >> roadmap?
> >>>>> If its not too far, we would like to make our contribution to ARIA
on
> >>>> UUID.
> >>>>>
> >>>>> Looking forward to your response.
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> /Vaish
> >>>>>
> >>>>> ________________________________
> >>>>> From: Avia Efrat <avia@cloudify.co>
> >>>>> Sent: Sunday, July 30, 2017 10:35:45 PM
> >>>>> To: dev@ariatosca.incubator.apache.org
> >>>>> Subject: Re: Unique identification of an instance element across
> >>> services
> >>>>>
> >>>>> First, good arguments from both 'sides'.
> >>>>>
> >>>>> I am for at least adding a uuid as an option, as ARIA is intended
to
> >> be
> >>>>> used at scale as well.
> >>>>> But currently, I am for the simple ids to be used as default (and
not
> >>>>> uuids). Like it or not, right now ARIA is more at a 'TOSCA
> >> playground'
> >>>>> stage, and I think that's perfectly fine =)
> >>>>> And at this stage, I think simple ids will be better, as they easier
> >> to
> >>>> use
> >>>>> via the CLI, but more importantly, don't clog the logs with long
> >>>>> meaningless strings. As ARIA matures, we could switch the default
to
> >>>> UUIDs.
> >>>>>
> >>>>> And BTW, as our log format is configurable, there could be other
ways
> >>>> than
> >>>>> UUIDs to distinguish between nodes with the 'same id' in a central
> >>>> logging
> >>>>> system, e.g using the user name as another indicator.
> >>>>>
> >>>>> On Thu, Jul 27, 2017 at 10:20 AM, Vaishnavi K.R <
> >>>>> vaishnavi.k.r@ericsson.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>>
> >>>>>> Thanks for the active discussion.
> >>>>>>
> >>>>>>
> >>>>>> Having UUID at the node instance level will just make the nodes
> >>> unique.
> >>>>>>
> >>>>>> And these names will not be used by the user directly as no
> >>> operations
> >>>>> are
> >>>>>> happening on the node instance name.
> >>>>>>
> >>>>>> But at the service template and the service level, UUID will
be of
> >>>> great
> >>>>>> help considering the multi user and multi tenancy situations.
> >>>>>>
> >>>>>>
> >>>>>> So in a large scale perspective, the node names and the service
> >>>> template
> >>>>>> names have high probability of being same.
> >>>>>>
> >>>>>> When these enter into the automated world, it will create more
> >>> problem
> >>>>>> when name conflicts occur and its adds overhead to make changes
> >> every
> >>>>> time
> >>>>>> to overcome the conflict.
> >>>>>>
> >>>>>>
> >>>>>> UUID at service template and the service level: will be of much
use
> >>> in
> >>>>> the
> >>>>>> above scenario and operations by user on these are less
> >>>>>>
> >>>>>> UUID at node instance level: makes the node much unique and
no
> >>>> operation
> >>>>>> happens on it
> >>>>>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> /Vaish
> >>>>>>
> >>>>>> ________________________________
> >>>>>> From: Tal Liron <tal@cloudify.co>
> >>>>>> Sent: Wednesday, July 26, 2017 8:48:40 PM
> >>>>>> To: dev@ariatosca.incubator.apache.org
> >>>>>> Subject: Re: Unique identification of an instance element across
> >>>> services
> >>>>>>
> >>>>>> I just don't see users having to deal much with node IDs outside
of
> >>>>> simple
> >>>>>> hello-world style tutorials, and I'd hate for the first impressions
> >>>> that
> >>>>>> users get out of ARIA is that it's just a playground for TOSCA.
It
> >>>> should
> >>>>>> be ready out-of-the-box for the real world.
> >>>>>>
> >>>>>> On Wed, Jul 26, 2017 at 9:13 AM, DeWayne Filppi <
> >> dewayne@cloudify.co
> >>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Such is their strength.  I'm just advocating using them
as a last
> >>>>> resort
> >>>>>>> because they are user unfriendly and gigantic.
> >>>>>>>
> >>>>>>> On Tue, Jul 25, 2017 at 12:55 PM, Tal Liron <tal@cloudify.co>
> >>> wrote:
> >>>>>>>
> >>>>>>>> Let's consider a mass deployment: thousands of service
> >> instances
> >>> of
> >>>>> the
> >>>>>>>> same service template, created by many different users
with
> >> their
> >>>> own
> >>>>>>> ARIA
> >>>>>>>> installations (and databases). In that case, assuming
we use
> >>>>> sequential
> >>>>>>>> IDs, you would have the same node ID appear many times.
You
> >> would
> >>>>> have
> >>>>>> to
> >>>>>>>> identify it via the particular user and service instance.
If
> >>> you're
> >>>>>>>> centralizing logs, this can quickly be cumbersome. A
UUID will
> >>>>> identify
> >>>>>>> it
> >>>>>>>> globally and avoid any confusion.
> >>>>>>>>
> >>>>>>>> I think the default should be something that avoids
such
> >>> problems.
> >>>>> For
> >>>>>>>> users who insist on shorter IDs, we can allow them to
configure
> >>> it.
> >>>>>>>>
> >>>>>>>> On Tue, Jul 25, 2017 at 2:42 PM, DeWayne Filppi <
> >>>> dewayne@cloudify.co
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> True uuids are seductive, because of their simplicity.
 But
> >>> they
> >>>>> are
> >>>>>>>> huge,
> >>>>>>>>> overkill, and meaningless.  Imho a structured id
is superior
> >> if
> >>>> it
> >>>>>> can
> >>>>>>> be
> >>>>>>>>> made to work without a global locking scheme.
> >>>>>>>>>
> >>>>>>>>> - DeWayne
> >>>>>>>>>
> >>>>>>>>> On Jul 25, 2017 12:11 PM, "Tal Liron" <tal@cloudify.co>
> >> wrote:
> >>>>>>>>>
> >>>>>>>>>> It's not an issue of thread safety -- it could
be entirely
> >>>>>> different
> >>>>>>>>>> processes, on different machines, accessing
the same db. It
> >>> can
> >>>>> be
> >>>>>>>> solved
> >>>>>>>>>> via a SQL transaction, but I feel the whole
issue can be
> >>>> avoided
> >>>>> by
> >>>>>>>> using
> >>>>>>>>>> UUIDs.
> >>>>>>>>>>
> >>>>>>>>>> Using the CLI to access specific nodes is not
something I
> >> see
> >>>>>>>> happening a
> >>>>>>>>>> lot outside of debugging. And when you do debug,
you'll
> >>>> probably
> >>>>> be
> >>>>>>>>> copying
> >>>>>>>>>> and pasting a node ID from the logs, so shorter
names do
> >> not
> >>>> add
> >>>>>> much
> >>>>>>>>> ease
> >>>>>>>>>> of use.
> >>>>>>>>>>
> >>>>>>>>>> Again, I would be personally happiest if this
was
> >>> configurable
> >>>>> (and
> >>>>>>>>>> personally think UUIDs should be the reasonable
default).
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jul 25, 2017 at 2:01 PM, Maxim Orlov
<
> >>>> maxim@cloudify.co>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Technically we have no issue with implementing
this via
> >>> uuid
> >>>>> or a
> >>>>>>>>>>> threadsafe solution for the current index
implementation.
> >>>>>>>>>>>
> >>>>>>>>>>> Getting node data via the cli feels more
intuitive using
> >>> the
> >>>>>> index
> >>>>>>>>> based
> >>>>>>>>>>> ID, rather than the uuid based ID in my
opionion.
> >>>>>>>>>>>
> >>>>>>>>>>> On Jul 25, 2017 9:49 PM, "Tal Liron" <tal@cloudify.co>
> >>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Our code for determining the next index
is not
> >> concurrently
> >>>>> safe
> >>>>>>> (no
> >>>>>>>>>> atomic
> >>>>>>>>>>> transaction) so I can see it breaking in
concurrent use
> >>> cases
> >>>>>>>> (running
> >>>>>>>>>> two
> >>>>>>>>>>> ARIA commands at the same time).
> >>>>>>>>>>>
> >>>>>>>>>>> What is to gain here in terms of human readability?
In my
> >>>>> opinion
> >>>>>>> it
> >>>>>>>>> adds
> >>>>>>>>>>> confusion because it gives a false sense
of
> >> predictability.
> >>>>>>>>>>>
> >>>>>>>>>>> In my opinion the best compromise is to
use
> >> base57-encoded
> >>>>> UUIDs.
> >>>>>>>> These
> >>>>>>>>>> are
> >>>>>>>>>>> true UUIDs, but use a mix of upper and lowercase
> >>>> alphanumerics
> >>>>>>>> ensuring
> >>>>>>>>>> no
> >>>>>>>>>>> visually ambiguous characters. We have the
code for this
> >> in
> >>>>>>>>>> utils/uuid.py.
> >>>>>>>>>>>
> >>>>>>>>>>> See also: https://github.com/wyattisimo/base57-ruby
> >>>>> [https://avatars1.githubusercontent.com/u/625546?v=3&s=400]<https://
> >>>>> github.com/wyattisimo/base57-ruby>
> >>>>>
> >>>>> GitHub - wyattisimo/base57-ruby: Base57 encoder for Ruby.<
> >>>>> https://github.com/wyattisimo/base57-ruby>
> >>>>> github.com
> >>>>> base57-ruby - Base57 encoder for Ruby. ... Clone with HTTPS Use
Git
> >> or
> >>>>> checkout with SVN using the web URL.
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jul 25, 2017 at 1:28 PM, Maxim Orlov
<
> >>>>> maxim@cloudify.co>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Actually the refactoring was made so
the id would be
> >> more
> >>>>> user
> >>>>>>>>>> readable.
> >>>>>>>>>>>> The index is determined according to
the used indices
> >>> (it's
> >>>>> not
> >>>>>>>> just
> >>>>>>>>> a
> >>>>>>>>>>>> running number). If indeed this poses
an issue (or if
> >>>> indeed
> >>>>> a
> >>>>>>> uuid
> >>>>>>>>> is
> >>>>>>>>>>>> easier to recognize, or even use in
a query), let's
> >>> discuss
> >>>>> it
> >>>>>>>>>> further...
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jul 25, 2017 at 7:35 PM, Tal
Liron <
> >>>> tal@cloudify.co>
> >>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> We used to use UUIDs but at some
point this was
> >>>>> refactored. I
> >>>>>>>> tend
> >>>>>>>>> to
> >>>>>>>>>>>> agree
> >>>>>>>>>>>>> with you.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Actually, I would prefer it to be
configurable. We
> >> have
> >>>>> code
> >>>>>> in
> >>>>>>>>> place
> >>>>>>>>>>> for
> >>>>>>>>>>>>> ID generation of various types:
UUIDs, short UUIDs,
> >> and
> >>>>>>>>> sequentials.
> >>>>>>>>>>> All
> >>>>>>>>>>>> of
> >>>>>>>>>>>>> them would seem useful to me for
various scenarios.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Jul 25, 2017 at 3:42 AM,
Vaishnavi K.R <
> >>>>>>>>>>>> vaishnavi.k.r@ericsson.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> With my understanding in current
ARIA, the node
> >>>> instances
> >>>>>> are
> >>>>>>>>> made
> >>>>>>>>>>>> unique
> >>>>>>>>>>>>>> by prefixing the node name with
the 'id of the
> >>> service'
> >>>>>> (i.e.
> >>>>>>>> the
> >>>>>>>>>>>> primary
> >>>>>>>>>>>>>> key of the service table) as
the instances are
> >>> specific
> >>>>> to
> >>>>>>> the
> >>>>>>>>>>> service.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What will be the name of the
node instances if the
> >>>>> default
> >>>>>>>>>> instances
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>> the node template is '3' and
how this will hold
> >> good
> >>>>> during
> >>>>>>>> scale
> >>>>>>>>>> in
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>> out?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Could UUID be of great help
in handling such cases
> >> by
> >>>>>>> including
> >>>>>>>>>> that
> >>>>>>>>>>>> as a
> >>>>>>>>>>>>>> column in the database tables
of the service and
> >> the
> >>>>> node?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This will wipe out the naming
confusions and
> >> querying
> >>>> can
> >>>>>> be
> >>>>>>>> made
> >>>>>>>>>>> easy
> >>>>>>>>>>>>>> with the UUIDs.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Looking forward to your suggestion.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> /Vaish
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message