incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Yadav <rohit.ya...@citrix.com>
Subject Re: Arch Rework RFC: API refactoring updates
Date Mon, 17 Dec 2012 18:57:07 GMT
From the last mail on this thread (http://markmail.org/thread/wueimbh6tyas4227), one can clearly
see I did not get it the first time, so last week I read a lot of code in cloud-api and cloud-server
to understand what's happening, how api server starts, loads stuff, handles requests etc.
and what to do about it. I think I can share what I understand now and what I've been dong
since Thursday; also so if anyone wants to chip in;

So, why we are doing, what we're doing in api_refactoring:

0. For better apidoc generation. Aggregate apis with src packages around common entity for
developers.
1. Decouple role based api call verification using a plugin/adapter.
2. Decouple tightly coupled database and acl validations from service layer to api layer using
@ACL, @Validator. They can be adapters too.
3. Using CommandType.UUID to annotate fields that are uuids from over the wire requests and
use this annotations to convert them from string to internal ID. entityType helps us know
the Response class whose @Entity helps us know the db table (impl. in EntityManagerImpl).
Useful for @Validator to validate db/range validations as well.
4. Remove bloat, decouple policy from mechanism (using plugins or adapters), data from code
wherever possible and use OOP styled coding, remove iterative/primitive C like code wherever
possible.
5. FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+API+refactoring

How it's done: (NOTE: Most of it would be done in all Cmd classes, only @Entity in Response
classes)

0. Remove IdentityMapper and do 1.
1. Fix/add entityType to a Response.class, in @Parameter. Goto 2.
2. For every Response class that was added in the entityType field in @Parameter in 1., add
@Entity(). The idea is that from this @Entity we would know the interface through which we
can know the db table. For. example for UserResponse, we should annotate with @Entity(User.class)
and which is an interfaces implemented by UserVO.java through which we know it has @Table(name="user").
This is useful to convert the over-the-wire requests that have uuids to convert them to internal
ids.

3. In @Parameter, if the entity would receive a uuid, change it's type to CommandType.UUID
(recently introduced) and collectionType (if it's collection of Long, or basically list of
uuids for example) too. This annotation will help us generate docs so users/developers would
know what params is actually uuid. (See 1-3, to better understand why we're doing this)

4. @ACL annotation for all params in a cmd class whose access control for the caller user
has to be checked. If the parameter is a Map (generally would be a Long that is UUID type),
a map or dictionary has a key and a value, we need to specify which one needs checking. For
example:

@ACL(checkValueAccess=true)
@Parameter(bla bla bla, entityType={class1Response.class,class2Response.class})
private Map IpToNetworkList;

5. Add @Validator (defined as @validate in FS) to annotate which field in Cmd needs to be
be validated and using which validators (current we've DBEntityValidator.class and RandgeValidator.class).
This is work in progress, won't work as of now. You may leave @Validator for now. Example:
@ACL
@Validator(validtors={DBEntityValidator.class})
@Parameter(bla bla bla, type=CommandType.UUID, entityType={class1Response.class,class2Response.class})
private Long domainId;

At present, all apis which are queried by passing a uuid (for ex. list apis with uuid param)
are broken on purpose (so it gets attention :). Work in progress.

Regards.

On 09-Dec-2012, at 5:41 PM, Mice Xia <mice_xia@tcloudcomputing.com> wrote:

> Alex,
> 
> Got it, very clear now and hope this clarification will be updated on the wiki.
> 
> Regards
> Mice
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com] 
> Sent: Saturday, December 08, 2012 8:47 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Arch Rework RFC: API refactoring updates
> 
> Mice,
> 
> Please take a look at my email reply to Dave.  There's no implied ACL in this separation.
> 
> --Alex
> 
>> -----Original Message-----
>> From: Mice Xia [mailto:mice_xia@tcloudcomputing.com]
>> Sent: Friday, December 07, 2012 4:36 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: RE: Arch Rework RFC: API refactoring updates
>> 
>> Fang,
>> 
>> I vote for the second way, to keep it simple and clear, each API 
>> server has the same binary artifacts and contains all API 
>> implementations, access control is based on RBAC/ACL, and 
>> blacklist(white List) can be defined on a per API server basis. And 
>> perhaps we don’t need to explicitly package classes by roles ( or security level),
this seems to hardcode APIs' ACL in the code level.
>> 
>> My 2 cent.
>> 
>> -Mice
>> 
>> -----Original Message-----
>> From: Fang Wang [mailto:fang.wang@citrix.com]
>> Sent: Saturday, December 08, 2012 7:44 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: RE: Arch Rework RFC: API refactoring updates
>> 
>> Hi Mice,
>> 
>> For now, we just separate the API into 2 roles: user and admin.
>> The admin can be separated further into root admin, domain admin, etc.
>> Further dynamic roles can be added.
>> Once the framework is there, we can further separate the admin roles.
>> 
>> Per your question, there can be 2 ways to handle the domain admin requests:
>> - one is like you mentioned, sysadmin has separate api servers to 
>> handle user and domain api requests separately.
>> - Another way is to deploy one api server, and the api server does the 
>> role base access check. So user cann't access the domain Admin apis., 
>> the access would be blocked and denied.  The advantage of this method is the flexibility.
>> E.g, add another new domain, or a new category admin, we can 
>> dynamically deploy and apply the roles.
>> 
>> 
>> Thanks,
>> -Fang
>> 
>> -----Original Message-----
>> From: Mice Xia [mailto:weiran.xia1@gmail.com]
>> Sent: Thursday, December 06, 2012 5:15 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: Arch Rework RFC: API refactoring updates
>> 
>> rohit,
>> 
>> i asked this question because requests from domain admins are usually 
>> from outside of datacenter for a public cloud.
>> 
>> so artifact built from user package can be directly deployed as a user 
>> api server and serve user requests. but to serve domain admin request, 
>> sysadmin has to prepare another api server with another set of apis, 
>> which is a sub set of admin package, is this correct?
>> 
>> sent from phone, sorry for typos.
>> 
>> mice
>> 
>> 在 2012年12月7日星期五,Rohit Yadav <rohit.yadav@citrix.com> 写道:
>>> 
>>> On 06-Dec-2012, at 4:29 PM, Mice Xia <mice_xia@tcloudcomputing.com>
>> wrote:
>>> 
>>>> [quote]
>>>> 1. Moved and classified all apis (except the anomalies, see other
>>>> email)
>> into namespace org.apache.cloudstack.api 2. There are two packages, 
>> admin and user in which the APIs are grouped based on security level. 
>> Note; APIs are not grouped based on roles.
>>>> - Because of 1,2,3; there would be two classes of API server. The 
>>>> first
>> one would be available for users, the user API server which would 
>> handle requests from network outside the datacenter which Alex refers 
>> to as "over the wire" requests. The second one would be available for 
>> sysadmins, the mgmt API server which would handle requests from 
>> admins/sysadmins within the datacenter;
>>>> [/quote]
>>>> 
>>>> I haven’t seen the codes, just one question.
>>>> If the intention is to deploy API server separately based on the
>> security level, where are APIs enabled for domain-admin located? In 
>> the user package or in the admin package?
>>> 
>>> The admin package. For domain-admin etc. the sysadmin would run mgmt
>> servers with different sets of apis s/he wants to provide for a domain admin.
>> There won't be any pre configured set of apis, this would allow a 
>> system admin to decide exactly what level of access s/he wants to 
>> provide to a domainadmin role user. This allows him/her to create 
>> multiple end points of mgmt server as well, just like the user api server.
>>> 
>>> Regards.
>>> 
>>>> 
>>>> Regards
>>>> Mice
>>>> 
>>>> -----Original Message-----
>>>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>>>> Sent: Friday, December 07, 2012 4:07 AM
>>>> To: cloudstack-dev@incubator.apache.org
>>>> Subject: Arch Rework RFC: API refactoring updates
>>>> 
>>>> Few updates;
>>>> 
>>>> 1. Moved and classified all apis (except the anomalies, see other
>>>> email)
>> into namespace org.apache.cloudstack.api 2. There are two packages, 
>> admin and user in which the APIs are grouped based on security level. 
>> Note; APIs are not grouped based on roles.
>>>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command
>>>> pkg,
>> the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
>>>> 4. Few usage related APIs could not be moved from cloud-server to
>> cloud-api, as they are tightly coupled with classes available in cloud-server.
>>>> 5. commands.properties is fixed.
>>>> 6. Checked, the api_refactoring merges fine with a minor merge
>> conflicts. The build on the branch is broken, would be fixed soon.
>>>> 
>>>> I request that if you're adding any new class, please use the 
>>>> namespace
>> org.apache.cloudstack.
>>>> 
>>>> Road ahead:
>>>> - Because of 1,2,3; there would be two classes of API server. The 
>>>> first
>> one would be available for users, the user API server which would 
>> handle requests from network outside the datacenter which Alex refers 
>> to as "over the wire" requests. The second one would be available for 
>> sysadmins, the mgmt API server which would handle requests from 
>> admins/sysadmins within the datacenter;
>>>> 
>>>> user API server would use only the org.apache.cloudstack.api.user
>> artifact admin API server would use both 
>> org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user 
>> Based on the arch. rework docs, ppts, talks, Alex's idea was to 
>> separate api based on security level would give ultimate isolation.
>>>> 
>>>> - Separate out cloud-api, and make it run as a separate web app 
>>>> like
>> awsapi.
>>>> - Annotations and separation of API based on api affinity would 
>>>> help
>> automate apidoc generation, api discovery over an api endpoint, so 
>> clients (UI or cloudmonkey etc.) would be able to discover
>>>> - The commands.properties syntax is horrible, I want to get rid of 
>>>> the
>> evil syntax by having an apiname annotation for all API Cmd classes 
>> and the API server would be able to load this info during runtime. In 
>> commands.properties, we should be just able to set policy for user 
>> role for each api, if apiname is not declared it's blacklisted. It 
>> becomes tricky with plugins. I don't know how to get it right the first time, but
let's try.
>>>> - ACL and security checking at API layer, the requests would be 
>>>> just
>> passed to (multiple) cloud-engine which won't handle them, as that 
>> will only orchestrate.
>>>> - Roles would be decided from commands.properties. Multiple API 
>>>> servers
>> can be run with different combination of rules in commands.properties. 
>> This would allow separation of policy from mechanism, and multiple 
>> roles and end points. For example, an admin can create http routers 
>> for these api server,
>> so: <url>/hr, <url>/finance, <url>/marketting can be proxies to

>> different user api servers with different set of whitelisted apis.
>>>> 
>>>> Comments, suggestions, flames?
>>>> 
>>>> Regards.
>>> 
>>> 
>> 
>> --
>> Sent from Gmail Mobile

Mime
View raw message