cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Weingärtner <rafaelweingart...@gmail.com>
Subject Re: Tag removal problem found. Need assistance how to fix
Date Wed, 22 Nov 2017 12:16:38 GMT
That is what it seems

On Wed, Nov 22, 2017 at 10:14 AM, Ivan Kudryavtsev <kudryavtsev_ia@bw-sw.com
> wrote:

> So, basically I changed it to
>
> sb.and("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN
> );
> sb.and("resourceType", sb.entity().getResourceType(),
> SearchCriteria.Op.EQ);
>
>
> And launched marvin tests. I suppose resourceId filtering is completely
> wrong here.
>
>
> 22 нояб. 2017 г. 7:12 ПП пользователь "Ivan Kudryavtsev" <
> kudryavtsev_ia@bw-sw.com> написал:
>
> Yes, you are right, but there are no other ids in the query, and rarely
> (like in my case) it leads to wrong results.
>
> 22 нояб. 2017 г. 7:07 ПП пользователь "Rafael Weingärtner" <
> rafaelweingartner@gmail.com> написал:
>
> The problem seems to be related to line 308 at
> > com.cloud.tags.TaggedResourceManagerImpl.deleteTags(List<String>,
> > ResourceObjectType, Map<String, String>).
> > It is being sent a list of resourceUUID as the filter for resourceId
> >
> > On Wed, Nov 22, 2017 at 10:03 AM, Rafael Weingärtner <
> > rafaelweingartner@gmail.com> wrote:
> >
> > > The resourceId is not a real DB ID. It is the UUID converted to Long
> > :(....
> > > This table has four "ID" like fields, ID, UUID, resourceID, and
> > > resourceUUID.
> > >
> > > On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> > > kudryavtsev_ia@bw-sw.com> wrote:
> > >
> > >> Hi, I just enabled log all queries and copied actual query which I
> have
> > >> shown in the first mail. Also, I don't understand why the search
> should
> > >> look over inner ids... Is it a case when user can pass real db ids to
> an
> > >> api call?
> > >>
> > >> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner"
<
> > >> rafaelweingartner@gmail.com> написал:
> > >>
> > >> > Yes, this I understood ;)
> > >> >
> > >> > However, I do not understand how the SQL that is being generated has
> > >> this
> > >> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > >> > c1bc5440ea60'".
> > >> > The resourceId field in the entity is a long. So, even though that
> > long
> > >> > represents a String, in the final SQL that is generated it should
> be a
> > >> long
> > >> > value there.
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> > >> > kudryavtsev_ia@bw-sw.com>
> > >> > wrote:
> > >> >
> > >> > > Take a look here:
> > >> > >
> > >> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > >> > > resource_tags.value, resource_tags.domain_id,
> > >> resource_tags.account_id,
> > >> > > resource_tags.resource_id, resource_tags.resource_uuid,
> > >> > > resource_tags.resource_type, resource_tags.customer FROM
> > resource_tags
> > >> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > >> > c1bc5440ea60'
> > >> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > >> > > 9465-c1bc5440ea60'
> > >> > > )  AND resource_tags.resource_type = 'Account';
> > >> > >
> > >> > > +----+--------------------------------------+-------+-------
> > >> > > +-----------+------------+-------------+--------------------
> > >> > > ------------------+---------------+----------+
> > >> > > | id | uuid                                 | key   | value |
> > >> domain_id |
> > >> > > account_id | resource_id | resource_uuid                    
   |
> > >> > > resource_type | customer |
> > >> > > +----+--------------------------------------+-------+-------
> > >> > > +-----------+------------+-------------+--------------------
> > >> > > ------------------+---------------+----------+
> > >> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me    |
> > >>  1
> > >> > |
> > >> > >        2 |           2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f
|
> > >> Account
> > >> > >   | NULL     |
> > >> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me    |
> > >>  1
> > >> > |
> > >> > >        4 |           4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60
|
> > >> Account
> > >> > >   | NULL     |
> > >> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> > >>  1
> > >> > |
> > >> > >        4 |           4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60
|
> > >> Account
> > >> > >   | NULL     |
> > >> > > +----+--------------------------------------+-------+-------
> > >> > > +-----------+------------+-------------+--------------------
> > >> > > ------------------+---------------+----------+
> > >> > > 3 rows in set, 1 warning (0.01 sec)
> > >> > >
> > >> > > Try to figure out why id=7 is selected here?
> > >> > >
> > >> > > Because:
> > >> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > >> > >
> > >> > > Matched unintentionally, because mysql converted uuid to int
and
> > got 2
> > >> > > which is matched to resource_id of 2 (id=7).
> > >> > >
> > >> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael
Weingärtner" <
> > >> > > rafaelweingartner@gmail.com> написал:
> > >> > >
> > >> > > > Ah, ok now it makes sense the "IN", I thought you were only
> > talking
> > >> > about
> > >> > > > single values.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > I do not think that the UUID (resource UUID) is the
> representation
> > >> of
> > >> > ID
> > >> > > > value in Hexadecimal, if it is we could simply get rid of
one of
> > >> them.
> > >> > I
> > >> > > > really dislike these search criteria...I am not seeing what
you
> > are
> > >> > > saying.
> > >> > > > Let´s see this in SQL, so we can discuss.
> > >> > > >
> > >> > > > SELECT * FROM resource_tags
> > >> > > > WHERE  (resource_id in (....) OR resource_uuid in (...))
> > >> > > > AND resource_tags.resource_type = 'Account';
> > >> > > >
> > >> > > >
> > >> > > > That is what the programmer who coded that Search criteria
> seemed
> > to
> > >> > > want,
> > >> > > > right? I mean, the developer wanted to select resources
that
> match
> > >> > either
> > >> > > > the ID or UUID field. Also, we may have more than a single
value
> > to
> > >> > > filter
> > >> > > > in both ID and UUID. I am also assuming that UUID does not
> > >> necessarily
> > >> > > > represents the ID as a hexadecimal.
> > >> > > >
> > >> > > > The problem seems to be when it is being translated:
> > >> > > >
> > >> > > > > ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> c1bc5440e
> > >> a60'
> > >> > > > > OR
> > >> > > > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > >> > > > 9465-c1bc5440ea60'
> > >> > > > > )
> > >> > > > >
> > >> > > >
> > >> > > > It is not even using an “IN” structure in the SQL. Also,
why is
> > the
> > >> > > > resource_id equals the UUID in the filter. Did you check
the
> > entity
> > >> > that
> > >> > > is
> > >> > > > being sent as an example? Are the fields ID and UUID set
with
> the
> > >> same
> > >> > > > values?
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > P.S. Normally the ID field of entities is not exposed to
users
> via
> > >> API.
> > >> > > The
> > >> > > > field ID in the API is translated to UUID in ACS. The field
ID
> in
> > >> the
> > >> > > > database is intended as a dummy primary key for the table.
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Nov 22, 2017 at 9:05 AM, Ivan Kudryavtsev <
> > >> > > > kudryavtsev_ia@bw-sw.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi, Rafael, 'IN" because API call assumes that several
> > resourceIds
> > >> > can
> > >> > > be
> > >> > > > > provided, so IN solves it, EQ doesn't.
> > >> > > > >
> > >> > > > > But despite semantics ID/UUID you see that ID is integer
and
> > UUID
> > >> is
> > >> > > > string
> > >> > > > > and that comparison does fault positive results, next
when
> > object
> > >> > > access
> > >> > > > > for caller is checked exception occured and no tag
removal
> > happen
> > >> as
> > >> > a
> > >> > > > > result because int(2) eq '2afcffdsfdsfds-... (UUID)".
> > >> > > > >
> > >> > > > > 2017-11-22 18:01 GMT+07:00 Rafael Weingärtner <
> > >> > > > rafaelweingartner@gmail.com
> > >> > > > > >:
> > >> > > > >
> > >> > > > > > Are ID and UUID set with the same values in that
entity? If
> > not,
> > >> > the
> > >> > > > > > criteria seem correct. I mean, it is trying to
filter for an
> > ID
> > >> if
> > >> > it
> > >> > > > > > exists or by UUID if it exists in the entity that
is passed
> as
> > >> an
> > >> > > > > example.
> > >> > > > > > What I do not understand is that they are using
“
> > >> > > SearchCriteria.Op.IN
> > >> > > > ”,
> > >> > > > > > but
> > >> > > > > > in my opinion, it should be “SearchCriteria.Op.EQ”.
> > >> > > > > >
> > >> > > > > > On Wed, Nov 22, 2017 at 7:58 AM, Ivan Kudryavtsev
<
> > >> > > > > > kudryavtsev_ia@bw-sw.com>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi, I found interesting behaviour with tags:
> > >> > > > > > >
> > >> > > > > > > mysql> SELECT resource_tags.id, resource_tags.uuid,
> > >> > > > resource_tags.key,
> > >> > > > > > > resource_tags.value, resource_tags.domain_id,
> > >> > > > resource_tags.account_id,
> > >> > > > > > > resource_tags.resource_id, resource_tags.resource_uuid,
> > >> > > > > > > resource_tags.resource_type, resource_tags.customer
FROM
> > >> > > > resource_tags
> > >> > > > > > > WHERE  ( resource_tags.resource_id='2a4
> > 264fb-9f63-4d4f-9465-
> > >> > > > > > c1bc5440ea60'
> > >> > > > > > > OR
> > >> > > > > > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > >> > > > > > 9465-c1bc5440ea60'
> > >> > > > > > > )
> > >> > > > > > >  AND resource_tags.resource_type = 'Account';
> > >> > > > > > >
> > >> > > > > > > +----+------------------------
> > --------------+-------+-------
> > >> > > > > > > +-----------+------------+----
> > ---------+--------------------
> > >> > > > > > > ------------------+---------------+----------+
> > >> > > > > > > | id | uuid                             
   | key   |
> value
> > |
> > >> > > > > domain_id |
> > >> > > > > > > account_id | resource_id | resource_uuid
> > >>   |
> > >> > > > > > > resource_type | customer |
> > >> > > > > > > +----+------------------------
> > --------------+-------+-------
> > >> > > > > > > +-----------+------------+----
> > ---------+--------------------
> > >> > > > > > > ------------------+---------------+----------+
> > >> > > > > > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1
| test  | me
> >   |
> > >> > > > >  1
> > >> > > > > > |
> > >> > > > > > >        2 |           2 | 3199fc71-cf39-11e7-af5d-dc0ea1
> > 6ecd7f
> > >> |
> > >> > > > > Account
> > >> > > > > > >   | NULL     |
> > >> > > > > > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9
| test3 | me
> >   |
> > >> > > > >  1
> > >> > > > > > |
> > >> > > > > > >        4 |           4 | 2a4264fb-9f63-4d4f-9465-c1bc54
> > 40ea60
> > >> |
> > >> > > > > Account
> > >> > > > > > >   | NULL     |
> > >> > > > > > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979
| test5 | me2
> >  |
> > >> > > > >  1
> > >> > > > > > |
> > >> > > > > > >        4 |           4 | 2a4264fb-9f63-4d4f-9465-c1bc54
> > 40ea60
> > >> |
> > >> > > > > Account
> > >> > > > > > >   | NULL     |
> > >> > > > > > > +----+------------------------
> > --------------+-------+-------
> > >> > > > > > > +-----------+------------+----
> > ---------+--------------------
> > >> > > > > > > ------------------+---------------+----------+
> > >> > > > > > > 3 rows in set, 1 warning (0.01 sec)
> > >> > > > > > >
> > >> > > > > > > Don't see that "resource_type" is "account".
I just play
> > with
> > >> it.
> > >> > > > > > >
> > >> > > > > > > Take a look at ID=7. This row is found because:
> > >> > > > > > >
> > >> > > > > > > resource_tags.resource_id='2a4
> > 264fb-9f63-4d4f-9465-c1bc5440e
> > >> a60'
> > >> > > > when
> > >> > > > > > > right
> > >> > > > > > > part is converted to int. Corresponding code
is here:
> > >> > > > > > >
> > >> > > > > > > https://github.com/apache/clou
> > dstack/blob/87ef8137534fa79810
> > >> > > > > 1f65c6691fcf
> > >> > > > > > > 71513ac978/server/src/com/clou
> > d/tags/TaggedResourceManagerIm
> > >> > > > > pl.java#L301
> > >> > > > > > >
> > >> > > > > > > sb.and().op("resourceId", sb.entity().getResourceId(),
> > >> > > > > > > SearchCriteria.Op.IN);
> > >> > > > > > > sb.or("resourceUuid", sb.entity().getResourceUuid(),
> > >> > > > > > SearchCriteria.Op.IN
> > >> > > > > > > );
> > >> > > > > > > sb.cp();
> > >> > > > > > > sb.and("resourceType", sb.entity().getResourceType(),
> > >> > > > > > > SearchCriteria.Op.EQ);
> > >> > > > > > >
> > >> > > > > > > I don't know why the writer uses "resourceId"
or
> > >> "resourceUuid".
> > >> > I
> > >> > > > > > suppose
> > >> > > > > > > it's a bug and code should be transformed
to:
> > >> > > > > > >
> > >> > > > > > > sb.and("resourceUuid", sb.entity().getResourceUuid(),
> > >> > > > > > SearchCriteria.Op.IN
> > >> > > > > > > );
> > >> > > > > > > sb.and("resourceType", sb.entity().getResourceType(),
> > >> > > > > > > SearchCriteria.Op.EQ);
> > >> > > > > > >
> > >> > > > > > > Or MySQL query should be transformed to:
> > >> > > > > > >
> > >> > > > > > > mysql> SELECT resource_tags.id, resource_tags.uuid,
> > >> > > > resource_tags.key,
> > >> > > > > > > resource_tags.value, resource_tags.domain_id,
> > >> > > > resource_tags.account_id,
> > >> > > > > > > resource_tags.resource_id, resource_tags.resource_uuid,
> > >> > > > > > > resource_tags.resource_type, resource_tags.customer
FROM
> > >> > > > resource_tags
> > >> > > > > > > WHERE  ( concat("%", resource_tags.resource_id)
=
> > >> > > > > > > '2a4264fb-9f63-4d4f-9465-c1bc5440ea60' OR
> > >> > > > > > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > >> > > > > > 9465-c1bc5440ea60'
> > >> > > > > > > )
> > >> > > > > > >  AND resource_tags.resource_type = 'Account';
> > >> > > > > > > +----+------------------------
> > --------------+-------+-------
> > >> > > > > > > +-----------+------------+----
> > ---------+--------------------
> > >> > > > > > > ------------------+---------------+----------+
> > >> > > > > > > | id | uuid                             
   | key   |
> value
> > |
> > >> > > > > domain_id |
> > >> > > > > > > account_id | resource_id | resource_uuid
> > >>   |
> > >> > > > > > > resource_type | customer |
> > >> > > > > > > +----+------------------------
> > --------------+-------+-------
> > >> > > > > > > +-----------+------------+----
> > ---------+--------------------
> > >> > > > > > > ------------------+---------------+----------+
> > >> > > > > > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9
| test3 | me
> >   |
> > >> > > > >  1
> > >> > > > > > |
> > >> > > > > > >        4 |           4 | 2a4264fb-9f63-4d4f-9465-c1bc54
> > 40ea60
> > >> |
> > >> > > > > Account
> > >> > > > > > >   | NULL     |
> > >> > > > > > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979
| test5 | me2
> >  |
> > >> > > > >  1
> > >> > > > > > |
> > >> > > > > > >        4 |           4 | 2a4264fb-9f63-4d4f-9465-c1bc54
> > 40ea60
> > >> |
> > >> > > > > Account
> > >> > > > > > >   | NULL     |
> > >> > > > > > > +----+------------------------
> > --------------+-------+-------
> > >> > > > > > > +-----------+------------+----
> > ---------+--------------------
> > >> > > > > > > ------------------+---------------+----------+
> > >> > > > > > > 2 rows in set (0.00 sec)
> > >> > > > > > >
> > >> > > > > > > Let me your thoughts and I'll fix it. Right
now, obviously
> > >> it's a
> > >> > > > bug.
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > --
> > >> > > > > > > With best regards, Ivan Kudryavtsev
> > >> > > > > > > Bitworks Software, Ltd.
> > >> > > > > > > Cell: +7-923-414-1515 <+7%20923%20414-15-15>
> > >> > > > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > > Rafael Weingärtner
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > With best regards, Ivan Kudryavtsev
> > >> > > > > Bitworks Software, Ltd.
> > >> > > > > Cell: +7-923-414-1515
> > >> > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Rafael Weingärtner
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Rafael Weingärtner
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Rafael Weingärtner
> > >
> >
> >
> >
> > --
> > Rafael Weingärtner
> >
>



-- 
Rafael Weingärtner

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message