Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A75FF200CC6 for ; Tue, 4 Jul 2017 03:06:20 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A5F79160BF9; Tue, 4 Jul 2017 01:06:20 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 56AC4160BEC for ; Tue, 4 Jul 2017 03:06:18 +0200 (CEST) Received: (qmail 31996 invoked by uid 500); 4 Jul 2017 01:06:17 -0000 Mailing-List: contact commits-help@ariatosca.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ariatosca.incubator.apache.org Delivered-To: mailing list commits@ariatosca.incubator.apache.org Received: (qmail 31987 invoked by uid 99); 4 Jul 2017 01:06:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Jul 2017 01:06:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E03071AFD7B for ; Tue, 4 Jul 2017 01:06:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.222 X-Spam-Level: X-Spam-Status: No, score=-4.222 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id lEPNDIh7-krM for ; Tue, 4 Jul 2017 01:06:05 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id A8C375F2F1 for ; Tue, 4 Jul 2017 01:06:04 +0000 (UTC) Received: (qmail 31683 invoked by uid 99); 4 Jul 2017 01:06:04 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Jul 2017 01:06:04 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 0CA64DFBA2; Tue, 4 Jul 2017 01:06:04 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: emblemparade@apache.org To: commits@ariatosca.incubator.apache.org Date: Tue, 04 Jul 2017 01:06:05 -0000 Message-Id: In-Reply-To: <4df8a4921a1f46b89968b416f4ff5f4a@git.apache.org> References: <4df8a4921a1f46b89968b416f4ff5f4a@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/2] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies archived-at: Tue, 04 Jul 2017 01:06:20 -0000 ARIA-254 Scaling capabilities and policies * New aria.Scaling policy (and "scaling" role) * NodeTemplate model no longer stores scaling values (default_instances, etc.) but instead fetches them from applicable scaling capabilities and policies * Some code cleanup Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/ee9c08a0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/ee9c08a0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/ee9c08a0 Branch: refs/heads/ARIA-254-multiple-nodes-per-template Commit: ee9c08a0a564e91429cdb46234a0b6a5e1dcfec7 Parents: 3583f8c Author: Tal Liron Authored: Thu Jun 1 14:17:17 2017 -0500 Committer: Tal Liron Committed: Mon Jul 3 20:05:08 2017 -0500 ---------------------------------------------------------------------- aria/cli/commands/workflows.py | 4 +- aria/cli/execution_logging.py | 4 +- aria/core.py | 2 +- aria/modeling/orchestration.py | 6 +- aria/modeling/relationship.py | 2 +- aria/modeling/service_instance.py | 134 ++------ aria/modeling/service_template.py | 336 ++++++++++--------- aria/modeling/types.py | 4 +- aria/orchestrator/workflow_runner.py | 2 +- aria/orchestrator/workflows/executor/celery.py | 4 +- aria/orchestrator/workflows/executor/process.py | 4 +- aria/orchestrator/workflows/executor/thread.py | 2 +- aria/parser/presentation/presentation.py | 12 +- aria/storage/core.py | 2 +- .../profiles/aria-1.0/aria-1.0.yaml | 38 ++- .../profiles/tosca-simple-1.0/capabilities.yaml | 1 + .../simple_v1_0/assignments.py | 8 + .../simple_v1_0/data_types.py | 22 +- .../simple_v1_0/definitions.py | 8 + .../aria_extension_tosca/simple_v1_0/filters.py | 2 + .../aria_extension_tosca/simple_v1_0/misc.py | 10 + .../simple_v1_0/modeling/__init__.py | 3 - .../simple_v1_0/modeling/artifacts.py | 4 +- .../simple_v1_0/modeling/capabilities.py | 48 ++- .../simple_v1_0/modeling/interfaces.py | 8 +- .../simple_v1_0/modeling/parameters.py | 5 +- .../simple_v1_0/presentation/extensible.py | 1 + .../presentation/field_validators.py | 20 ++ .../simple_v1_0/presentation/types.py | 2 + .../simple_v1_0/presenter.py | 3 +- .../simple_v1_0/templates.py | 8 +- .../aria_extension_tosca/simple_v1_0/types.py | 10 +- tests/end2end/test_hello_world.py | 2 +- tests/end2end/test_nodecellar.py | 2 +- tests/mock/models.py | 8 +- tests/modeling/test_mixins.py | 3 - tests/modeling/test_models.py | 20 +- tests/orchestrator/context/test_operation.py | 2 +- .../node-cellar/node-cellar.yaml | 21 +- .../node-cellar/types/nodejs.yaml | 1 + .../node-cellar/types/openstack.yaml | 3 +- 41 files changed, 440 insertions(+), 341 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/cli/commands/workflows.py ---------------------------------------------------------------------- diff --git a/aria/cli/commands/workflows.py b/aria/cli/commands/workflows.py index 03cf00e..ca191aa 100644 --- a/aria/cli/commands/workflows.py +++ b/aria/cli/commands/workflows.py @@ -50,7 +50,7 @@ def show(workflow_name, service_name, model_storage, logger): logger.info('Retrieving workflow {0} for service {1}'.format( workflow_name, service_name)) service = model_storage.service.get_by_name(service_name) - workflow = next((wf for wf in service.workflows.values() if + workflow = next((wf for wf in service.workflows.itervalues() if wf.name == workflow_name), None) if not workflow: raise AriaCliError( @@ -102,7 +102,7 @@ def list(service_name, model_storage, logger): """ logger.info('Listing workflows for service {0}...'.format(service_name)) service = model_storage.service.get_by_name(service_name) - workflows_list = sorted(service.workflows.values(), key=lambda w: w.name) + workflows_list = sorted(service.workflows.itervalues(), key=lambda w: w.name) defaults = { 'service_template_name': service.service_template_name, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/cli/execution_logging.py ---------------------------------------------------------------------- diff --git a/aria/cli/execution_logging.py b/aria/cli/execution_logging.py index af40e01..915038b 100644 --- a/aria/cli/execution_logging.py +++ b/aria/cli/execution_logging.py @@ -111,11 +111,11 @@ def stylize_log(item, mark_pattern): if item.task: # operation task implementation = item.task.function - inputs = dict(arg.unwrapped for arg in item.task.arguments.values()) + inputs = dict(arg.unwrapped for arg in item.task.arguments.itervalues()) else: # execution task implementation = item.execution.workflow_name - inputs = dict(inp.unwrapped for inp in item.execution.inputs.values()) + inputs = dict(inp.unwrapped for inp in item.execution.inputs.itervalues()) stylized_str = color.StringStylizer(_get_format()) _populate_level(stylized_str, item) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/core.py ---------------------------------------------------------------------- diff --git a/aria/core.py b/aria/core.py index a8d5245..e214b1a 100644 --- a/aria/core.py +++ b/aria/core.py @@ -110,7 +110,7 @@ class Core(object): 'Active execution ID: {1}'.format(service.name, active_executions[0].id)) if not force: - available_nodes = [str(n.id) for n in service.nodes.values() if n.is_available()] + available_nodes = [str(n.id) for n in service.nodes.itervalues() if n.is_available()] if available_nodes: raise exceptions.DependentAvailableNodesError( 'Can\'t delete service `{0}` - there are available nodes for this service. ' http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/modeling/orchestration.py ---------------------------------------------------------------------- diff --git a/aria/modeling/orchestration.py b/aria/modeling/orchestration.py index 7068557..df2643e 100644 --- a/aria/modeling/orchestration.py +++ b/aria/modeling/orchestration.py @@ -220,8 +220,10 @@ class TaskBase(mixins.ModelMixin): __tablename__ = 'task' - __private_fields__ = ('dependency_operation_task_fk', 'dependency_stub_task_fk', 'node_fk', - 'relationship_fk', 'plugin_fk', 'execution_fk') + __private_fields__ = ('node_fk', + 'relationship_fk', + 'plugin_fk', + 'execution_fk') START_WORKFLOW = 'start_workflow' END_WORKFLOW = 'end_workflow' http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/modeling/relationship.py ---------------------------------------------------------------------- diff --git a/aria/modeling/relationship.py b/aria/modeling/relationship.py index 8b6028f..0d906de 100644 --- a/aria/modeling/relationship.py +++ b/aria/modeling/relationship.py @@ -374,7 +374,7 @@ def _get_class_for_table(model_class, tablename): if tablename in (model_class.__name__, model_class.__tablename__): return model_class - for table_cls in model_class._decl_class_registry.values(): + for table_cls in model_class._decl_class_registry.itervalues(): if tablename == getattr(table_cls, '__tablename__', None): return table_cls http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/modeling/service_instance.py ---------------------------------------------------------------------- diff --git a/aria/modeling/service_instance.py b/aria/modeling/service_instance.py index 002a87c..9743847 100644 --- a/aria/modeling/service_instance.py +++ b/aria/modeling/service_instance.py @@ -58,14 +58,6 @@ class ServiceBase(InstanceModelMixin): __private_fields__ = ('substitution_fk', 'service_template_fk') - # region association proxies - - @declared_attr - def service_template_name(cls): - return relationship.association_proxy('service_template', 'name', type=':obj:`basestring`') - - # endregion - # region one_to_one relationships @declared_attr @@ -200,6 +192,14 @@ class ServiceBase(InstanceModelMixin): # endregion + # region association proxies + + @declared_attr + def service_template_name(cls): + return relationship.association_proxy('service_template', 'name', type=':obj:`basestring`') + + # endregion + # region foreign keys @declared_attr @@ -383,7 +383,7 @@ class NodeBase(InstanceModelMixin): DELETED = 'deleted' ERROR = 'error' - # 'deleted' isn't actually part of the TOSCA spec, since according the description of the + # Note: 'deleted' isn't actually part of the TOSCA spec, since according the description of the # 'deleting' state: "Node is transitioning from its current state to one where it is deleted and # its state is no longer tracked by the instance model." However, we prefer to be able to # retrieve information about deleted nodes, so we chose to add this 'deleted' state to enable us @@ -398,18 +398,6 @@ class NodeBase(InstanceModelMixin): 'stop': {'transitional': STOPPING, 'finished': CONFIGURED}, 'delete': {'transitional': DELETING, 'finished': DELETED}} - # region association proxies - - @declared_attr - def service_name(cls): - return relationship.association_proxy('service', 'name', type=':obj:`basestring`') - - @declared_attr - def node_template_name(cls): - return relationship.association_proxy('node_template', 'name', type=':obj:`basestring`') - - # endregion - # region one_to_one relationships @declared_attr @@ -545,6 +533,18 @@ class NodeBase(InstanceModelMixin): # endregion + # region association proxies + + @declared_attr + def service_name(cls): + return relationship.association_proxy('service', 'name', type=':obj:`basestring`') + + @declared_attr + def node_template_name(cls): + return relationship.association_proxy('node_template', 'name', type=':obj:`basestring`') + + # endregion + # region foreign_keys @declared_attr @@ -794,14 +794,6 @@ class GroupBase(InstanceModelMixin): 'service_fk', 'group_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -940,14 +932,6 @@ class PolicyBase(InstanceModelMixin): 'service_fk', 'policy_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1082,14 +1066,6 @@ class SubstitutionBase(InstanceModelMixin): __private_fields__ = ('node_type_fk', 'substitution_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1184,13 +1160,9 @@ class SubstitutionMappingBase(InstanceModelMixin): __tablename__ = 'substitution_mapping' __private_fields__ = ('substitution_fk', + 'node_fk', 'capability_fk', - 'requirement_template_fk', - 'node_fk') - - # region association proxies - - # endregion + 'requirement_template_fk') # region one_to_one relationships @@ -1224,10 +1196,6 @@ class SubstitutionMappingBase(InstanceModelMixin): # endregion - # region one_to_many relationships - - # endregion - # region many_to_one relationships @declared_attr @@ -1315,18 +1283,6 @@ class RelationshipBase(InstanceModelMixin): 'target_position', 'source_position') - # region association proxies - - @declared_attr - def source_node_name(cls): - return relationship.association_proxy('source_node', 'name') - - @declared_attr - def target_node_name(cls): - return relationship.association_proxy('target_node', 'name') - - # endregion - # region one_to_one relationships @declared_attr @@ -1422,6 +1378,18 @@ class RelationshipBase(InstanceModelMixin): # endregion + # region association proxies + + @declared_attr + def source_node_name(cls): + return relationship.association_proxy('source_node', 'name') + + @declared_attr + def target_node_name(cls): + return relationship.association_proxy('target_node', 'name') + + # endregion + # region foreign keys @declared_attr @@ -1526,14 +1494,6 @@ class CapabilityBase(InstanceModelMixin): 'node_fk', 'capability_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1672,14 +1632,6 @@ class InterfaceBase(InstanceModelMixin): 'relationship_fk', 'interface_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1844,10 +1796,6 @@ class OperationBase(InstanceModelMixin): 'plugin_fk', 'operation_template_fk') - # region association proxies - - # endregion - # region one_to_one relationships @declared_attr @@ -1923,10 +1871,6 @@ class OperationBase(InstanceModelMixin): # endregion - # region many_to_many relationships - - # endregion - # region foreign_keys @declared_attr @@ -2102,14 +2046,6 @@ class ArtifactBase(InstanceModelMixin): 'node_fk', 'artifact_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/modeling/service_template.py ---------------------------------------------------------------------- diff --git a/aria/modeling/service_template.py b/aria/modeling/service_template.py index 57fd672..df2f1a0 100644 --- a/aria/modeling/service_template.py +++ b/aria/modeling/service_template.py @@ -65,34 +65,6 @@ class ServiceTemplateBase(TemplateModelMixin): 'interface_type_fk', 'artifact_type_fk') - description = Column(Text, doc=""" - Human-readable description. - - :type: :obj:`basestring` - """) - - main_file_name = Column(Text, doc=""" - Filename of CSAR or YAML file from which this service template was parsed. - - :type: :obj:`basestring` - """) - - created_at = Column(DateTime, nullable=False, index=True, doc=""" - Creation timestamp. - - :type: :class:`~datetime.datetime` - """) - - updated_at = Column(DateTime, doc=""" - Update timestamp. - - :type: :class:`~datetime.datetime` - """) - - # region association proxies - - # endregion - # region one_to_one relationships @declared_attr @@ -253,10 +225,6 @@ class ServiceTemplateBase(TemplateModelMixin): # endregion - # region many_to_one relationships - - # endregion - # region many_to_many relationships @declared_attr @@ -315,6 +283,30 @@ class ServiceTemplateBase(TemplateModelMixin): # endregion + description = Column(Text, doc=""" + Human-readable description. + + :type: :obj:`basestring` + """) + + main_file_name = Column(Text, doc=""" + Filename of CSAR or YAML file from which this service template was parsed. + + :type: :obj:`basestring` + """) + + created_at = Column(DateTime, nullable=False, index=True, doc=""" + Creation timestamp. + + :type: :class:`~datetime.datetime` + """) + + updated_at = Column(DateTime, doc=""" + Update timestamp. + + :type: :class:`~datetime.datetime` + """) + @property def as_raw(self): return collections.OrderedDict(( @@ -341,12 +333,14 @@ class ServiceTemplateBase(TemplateModelMixin): def instantiate(self, container, model_storage, inputs=None): # pylint: disable=arguments-differ from . import models - context = ConsumptionContext.get_thread_local() now = datetime.now() service = models.Service(created_at=now, updated_at=now, description=deepcopy_with_locators(self.description), service_template=self) + + # TODO: we want to remove this use of the context + context = ConsumptionContext.get_thread_local() context.modeling.instance = service service.inputs = utils.merge_parameter_values(inputs, self.inputs, model_cls=models.Input) @@ -365,7 +359,7 @@ class ServiceTemplateBase(TemplateModelMixin): utils.instantiate_dict(self, service.meta_data, self.meta_data) for node_template in self.node_templates.itervalues(): - for _ in range(node_template.default_instances): + for _ in range(node_template.scaling['default_instances']): node = node_template.instantiate(container) service.nodes[node.name] = node @@ -468,22 +462,6 @@ class NodeTemplateBase(TemplateModelMixin): __private_fields__ = ('type_fk', 'service_template_fk') - # region association proxies - - @declared_attr - def service_template_name(cls): - return relationship.association_proxy('service_template', 'name') - - @declared_attr - def type_name(cls): - return relationship.association_proxy('type', 'name') - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -573,6 +551,18 @@ class NodeTemplateBase(TemplateModelMixin): # endregion + # region association proxies + + @declared_attr + def service_template_name(cls): + return relationship.association_proxy('service_template', 'name') + + @declared_attr + def type_name(cls): + return relationship.association_proxy('type', 'name') + + # endregion + # region foreign_keys @declared_attr @@ -617,22 +607,12 @@ class NodeTemplateBase(TemplateModelMixin): :type: [:class:`NodeTemplateConstraint`] """) - def is_target_node_template_valid(self, target_node_template): - if self.target_node_template_constraints: - for node_template_constraint in self.target_node_template_constraints: - if not node_template_constraint.matches(self, target_node_template): - return False - return True - @property def as_raw(self): return collections.OrderedDict(( ('name', self.name), ('description', self.description), ('type_name', self.type.name), - ('default_instances', self.default_instances), - ('min_instances', self.min_instances), - ('max_instances', self.max_instances), ('properties', formatting.as_raw_dict(self.properties)), ('attributes', formatting.as_raw_dict(self.properties)), ('interface_templates', formatting.as_raw_list(self.interface_templates)), @@ -642,13 +622,7 @@ class NodeTemplateBase(TemplateModelMixin): def instantiate(self, container): from . import models - if self.nodes: - highest_name_suffix = max(int(n.name.rsplit('_', 1)[-1]) for n in self.nodes) - suffix = highest_name_suffix + 1 - else: - suffix = 1 - name = '{name}_{index}'.format(name=self.name, index=suffix) - node = models.Node(name=name, + node = models.Node(name=self.next_name, type=self.type, description=deepcopy_with_locators(self.description), state=models.Node.INITIAL, @@ -660,10 +634,12 @@ class NodeTemplateBase(TemplateModelMixin): utils.instantiate_dict(node, node.capabilities, self.capability_templates) # Default attributes - if 'tosca_name' in node.attributes: + if ('tosca_name' in node.attributes) \ + and (node.attributes['tosca_name'].type_name == 'string'): node.attributes['tosca_name'].value = self.name - if 'tosca_id' in node.attributes: - node.attributes['tosca_id'].value = name + if 'tosca_id' in node.attributes \ + and (node.attributes['tosca_id'].type_name == 'string'): + node.attributes['tosca_id'].value = node.name return node @@ -690,12 +666,6 @@ class NodeTemplateBase(TemplateModelMixin): console.puts(context.style.meta(self.description)) with context.style.indent: console.puts('Type: {0}'.format(context.style.type(self.type.name))) - console.puts('Instances: {0:d} ({1:d}{2})'.format( - self.default_instances, - self.min_instances, - ' to {0:d}'.format(self.max_instances) - if self.max_instances is not None - else ' or more')) utils.dump_dict_values(self.properties, 'Properties') utils.dump_dict_values(self.attributes, 'Attributes') utils.dump_interfaces(self.interface_templates) @@ -703,6 +673,97 @@ class NodeTemplateBase(TemplateModelMixin): utils.dump_dict_values(self.capability_templates, 'Capability templates') utils.dump_list_values(self.requirement_templates, 'Requirement templates') + @property + def next_index(self): + """ + Next available node index. + + :returns: node index + :rtype: int + """ + + max_index = 0 + if self.nodes: + max_index = max(int(n.name.rsplit('_', 1)[-1]) for n in self.nodes) + return max_index + 1 + + @property + def next_name(self): + """ + Next available node name. + + :returns: node name + :rtype: basestring + """ + + return '{name}_{index}'.format(name=self.name, index=self.next_index) + + @property + def scaling(self): + scaling = {} + + def extract_property(properties, name): + if name in scaling: + return + prop = properties.get(name) + if (prop is not None) and (prop.type_name == 'integer') and (prop.value is not None): + scaling[name] = prop.value + + def extract_properties(properties): + extract_property(properties, 'min_instances') + extract_property(properties, 'max_instances') + extract_property(properties, 'default_instances') + + def default_property(name, value): + if name not in scaling: + scaling[name] = value + + # From our scaling capabilities + for capability_template in self.capability_templates.itervalues(): + if capability_template.type.role == 'scaling': + extract_properties(capability_template.properties) + + # From service scaling policies + for policy_template in self.service_template.policy_templates.itervalues(): + if policy_template.type.role == 'scaling': + if policy_template.is_for_node_template(self.name): + extract_properties(policy_template.properties) + + # Defaults + default_property('min_instances', 0) + default_property('max_instances', 1) + default_property('default_instances', 1) + + # Validate + # pylint: disable=too-many-boolean-expressions + if (scaling['min_instances'] < 0) or \ + (scaling['max_instances'] < 0) or \ + (scaling['default_instances'] < 0) or \ + (scaling['max_instances'] < scaling['min_instances']) or \ + (scaling['default_instances'] < scaling['min_instances']) or \ + (scaling['default_instances'] > scaling['max_instances']): + context = ConsumptionContext.get_thread_local() + context.validation.report('invalid scaling parameters for node template "{0}": ' + 'min={1}, max={2}, default={3}'.format( + self.name, + scaling['min_instances'], + scaling['max_instances'], + scaling['default_instances']), + level=validation.Issue.BETWEEN_TYPES) + + return scaling + + def is_target_node_template_valid(self, target_node_template): + """ + Checks if ``target_node_template`` matches all our ``target_node_template_constraints``. + """ + + if self.target_node_template_constraints: + for node_template_constraint in self.target_node_template_constraints: + if not node_template_constraint.matches(self, target_node_template): + return False + return True + class GroupTemplateBase(TemplateModelMixin): """ @@ -715,14 +776,6 @@ class GroupTemplateBase(TemplateModelMixin): __private_fields__ = ('type_fk', 'service_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -852,6 +905,12 @@ class GroupTemplateBase(TemplateModelMixin): console.puts('Member node templates: {0}'.format(', '.join( (str(context.style.node(v.name)) for v in self.node_templates)))) + def contains_node_template(self, name): + for node_template in self.node_templates: + if node_template.name == name: + return True + return False + class PolicyTemplateBase(TemplateModelMixin): """ @@ -864,14 +923,6 @@ class PolicyTemplateBase(TemplateModelMixin): __private_fields__ = ('type_fk', 'service_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1002,6 +1053,21 @@ class PolicyTemplateBase(TemplateModelMixin): console.puts('Target group templates: {0}'.format(', '.join( (str(context.style.node(v.name)) for v in self.group_templates)))) + def is_for_node_template(self, name): + for node_template in self.node_templates: + if node_template.name == name: + return True + for group_template in self.group_templates: + if group_template.contains_node_template(name): + return True + return False + + def is_for_group_template(self, name): + for group_template in self.group_templates: + if group_template.name == name: + return True + return False + class SubstitutionTemplateBase(TemplateModelMixin): """ @@ -1013,14 +1079,6 @@ class SubstitutionTemplateBase(TemplateModelMixin): __private_fields__ = ('node_type_fk',) - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1109,10 +1167,6 @@ class SubstitutionTemplateMappingBase(TemplateModelMixin): 'capability_template_fk', 'requirement_template_fk') - # region association proxies - - # endregion - # region one_to_one relationships @declared_attr @@ -1137,10 +1191,6 @@ class SubstitutionTemplateMappingBase(TemplateModelMixin): # endregion - # region one_to_many relationships - - # endregion - # region many_to_one relationships @declared_attr @@ -1255,10 +1305,6 @@ class RequirementTemplateBase(TemplateModelMixin): 'relationship_template_fk', 'node_template_fk') - # region association proxies - - # endregion - # region one_to_one relationships @declared_attr @@ -1399,7 +1445,7 @@ class RequirementTemplateBase(TemplateModelMixin): # Find first node that matches the type elif self.target_node_type is not None: for target_node_template in \ - self.node_template.service_template.node_templates.values(): + self.node_template.service_template.node_templates.itervalues(): if self.target_node_type.get_descendant(target_node_template.type.name) is None: continue @@ -1487,14 +1533,6 @@ class RelationshipTemplateBase(TemplateModelMixin): __private_fields__ = ('type_fk',) - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1607,14 +1645,6 @@ class CapabilityTemplateBase(TemplateModelMixin): __private_fields__ = ('type_fk', 'node_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1791,14 +1821,6 @@ class InterfaceTemplateBase(TemplateModelMixin): 'group_template_fk', 'relationship_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -1952,10 +1974,6 @@ class OperationTemplateBase(TemplateModelMixin): 'interface_template_fk', 'plugin_fk') - # region association proxies - - # endregion - # region one_to_one relationships @declared_attr @@ -2024,10 +2042,6 @@ class OperationTemplateBase(TemplateModelMixin): # endregion - # region many_to_many relationships - - # endregion - # region foreign keys @declared_attr @@ -2176,14 +2190,6 @@ class ArtifactTemplateBase(TemplateModelMixin): __private_fields__ = ('type_fk', 'node_template_fk') - # region association proxies - - # endregion - - # region one_to_one relationships - - # endregion - # region one_to_many relationships @declared_attr @@ -2334,18 +2340,6 @@ class PluginSpecificationBase(TemplateModelMixin): __private_fields__ = ('service_template_fk', 'plugin_fk') - version = Column(Text, doc=""" - Minimum plugin version. - - :type: :obj:`basestring` - """) - - enabled = Column(Boolean, nullable=False, default=True, doc=""" - Whether the plugin is enabled. - - :type: :obj:`bool` - """) - # region many_to_one relationships @declared_attr @@ -2382,6 +2376,18 @@ class PluginSpecificationBase(TemplateModelMixin): # endregion + version = Column(Text, doc=""" + Minimum plugin version. + + :type: :obj:`basestring` + """) + + enabled = Column(Boolean, nullable=False, default=True, doc=""" + Whether the plugin is enabled. + + :type: :obj:`bool` + """) + @property def as_raw(self): return collections.OrderedDict(( http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/modeling/types.py ---------------------------------------------------------------------- diff --git a/aria/modeling/types.py b/aria/modeling/types.py index c34326e..38240fa 100644 --- a/aria/modeling/types.py +++ b/aria/modeling/types.py @@ -287,10 +287,10 @@ JSON-serializable strict list type for SQLAlchemy columns. def _mutable_association_listener(mapper, cls): strict_dict_type_to_listener = \ - dict((v.type_cls, v.listener_cls) for v in _StrictDict._strict_map.values()) + dict((v.type_cls, v.listener_cls) for v in _StrictDict._strict_map.itervalues()) strict_list_type_to_listener = \ - dict((v.type_cls, v.listener_cls) for v in _StrictList._strict_map.values()) + dict((v.type_cls, v.listener_cls) for v in _StrictList._strict_map.itervalues()) for prop in mapper.column_attrs: column_type = prop.columns[0].type http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/orchestrator/workflow_runner.py ---------------------------------------------------------------------- diff --git a/aria/orchestrator/workflow_runner.py b/aria/orchestrator/workflow_runner.py index df1725f..47270c0 100644 --- a/aria/orchestrator/workflow_runner.py +++ b/aria/orchestrator/workflow_runner.py @@ -93,7 +93,7 @@ class WorkflowRunner(object): executor = executor or ProcessExecutor(plugin_manager=plugin_manager) # transforming the execution inputs to dict, to pass them to the workflow function - execution_inputs_dict = dict(inp.unwrapped for inp in self.execution.inputs.values()) + execution_inputs_dict = dict(inp.unwrapped for inp in self.execution.inputs.itervalues()) if not self._is_resume: workflow_fn = self._get_workflow_fn() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/orchestrator/workflows/executor/celery.py ---------------------------------------------------------------------- diff --git a/aria/orchestrator/workflows/executor/celery.py b/aria/orchestrator/workflows/executor/celery.py index 0716e5b..a2b3513 100644 --- a/aria/orchestrator/workflows/executor/celery.py +++ b/aria/orchestrator/workflows/executor/celery.py @@ -44,12 +44,12 @@ class CeleryExecutor(BaseExecutor): def _execute(self, ctx): self._tasks[ctx.id] = ctx - arguments = dict(arg.unwrapped for arg in ctx.arguments.values()) + arguments = dict(arg.unwrapped for arg in ctx.task.arguments.itervalues()) arguments['ctx'] = ctx.context self._results[ctx.id] = self._app.send_task( ctx.operation_mapping, kwargs=arguments, - task_id=ctx.id, + task_id=ctx.task.id, queue=self._get_queue(ctx)) def close(self): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/orchestrator/workflows/executor/process.py ---------------------------------------------------------------------- diff --git a/aria/orchestrator/workflows/executor/process.py b/aria/orchestrator/workflows/executor/process.py index 81da26f..185f15f 100644 --- a/aria/orchestrator/workflows/executor/process.py +++ b/aria/orchestrator/workflows/executor/process.py @@ -171,9 +171,9 @@ class ProcessExecutor(base.BaseExecutor): return { 'task_id': ctx.task.id, 'function': ctx.task.function, - 'operation_arguments': dict(arg.unwrapped for arg in ctx.task.arguments.values()), + 'operation_arguments': dict(arg.unwrapped for arg in ctx.task.arguments.itervalues()), 'port': self._server_port, - 'context': ctx.serialization_dict, + 'context': ctx.serialization_dict } def _construct_subprocess_env(self, task): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/orchestrator/workflows/executor/thread.py ---------------------------------------------------------------------- diff --git a/aria/orchestrator/workflows/executor/thread.py b/aria/orchestrator/workflows/executor/thread.py index d9dcdf8..26484dc 100644 --- a/aria/orchestrator/workflows/executor/thread.py +++ b/aria/orchestrator/workflows/executor/thread.py @@ -63,7 +63,7 @@ class ThreadExecutor(BaseExecutor): self._task_started(ctx) try: task_func = imports.load_attribute(ctx.task.function) - arguments = dict(arg.unwrapped for arg in ctx.task.arguments.values()) + arguments = dict(arg.unwrapped for arg in ctx.task.arguments.itervalues()) task_func(ctx=ctx, **arguments) self._task_succeeded(ctx) except BaseException as e: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/parser/presentation/presentation.py ---------------------------------------------------------------------- diff --git a/aria/parser/presentation/presentation.py b/aria/parser/presentation/presentation.py index 7292562..fe55b05 100644 --- a/aria/parser/presentation/presentation.py +++ b/aria/parser/presentation/presentation.py @@ -34,6 +34,14 @@ class Value(object): self.value = deepcopy_with_locators(value) self.description = deepcopy_with_locators(description) + def _dump(self, context): + if self.type is not None: + puts(context.style.type(self.type)) + if self.value is not None: + puts(context.style.literal(self.value)) + if self.description is not None: + puts(context.style.meta(self.description)) + class PresentationBase(HasCachedMethods): """ @@ -230,6 +238,8 @@ class AsIsPresentation(PresentationBase): def _dump(self, context): if hasattr(self._raw, '_dump'): - self._raw._dump(context) + puts(context.style.node(self._name)) + with context.style.indent: + self._raw._dump(context) else: super(AsIsPresentation, self)._dump(context) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/aria/storage/core.py ---------------------------------------------------------------------- diff --git a/aria/storage/core.py b/aria/storage/core.py index 74b1147..2a5745e 100644 --- a/aria/storage/core.py +++ b/aria/storage/core.py @@ -143,7 +143,7 @@ class ModelStorage(Storage): """ Drop all the tables. """ - for mapi in self.registered.values(): + for mapi in self.registered.itervalues(): mapi.drop() @contextmanager http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/profiles/aria-1.0/aria-1.0.yaml ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/profiles/aria-1.0/aria-1.0.yaml b/extensions/aria_extension_tosca/profiles/aria-1.0/aria-1.0.yaml index c1dc11d..e421150 100644 --- a/extensions/aria_extension_tosca/profiles/aria-1.0/aria-1.0.yaml +++ b/extensions/aria_extension_tosca/profiles/aria-1.0/aria-1.0.yaml @@ -58,4 +58,40 @@ policy_types: the full path to a Python @workflow function that generates a task graph based on the service topology. type: string - required: true + + aria.Scaling: + _extensions: + type_qualified_name: aria:Scaling + role: scaling + description: >- + Scaling. + derived_from: tosca.policies.Scaling + properties: + min_instances: + description: >- + This property is used to indicate the minimum number of instances that should be created + for the associated TOSCA Node Template by a TOSCA orchestrator. + type: integer + default: 1 + constraints: + - greater_or_equal: 0 + max_instances: + description: >- + This property is used to indicate the maximum number of instances that should be created + for the associated TOSCA Node Template by a TOSCA orchestrator. + type: integer + default: 1 + constraints: + - greater_or_equal: 0 + default_instances: + description: >- + An optional property that indicates the requested default number of instances that should + be the starting number of instances a TOSCA orchestrator should attempt to allocate. Note: + The value for this property MUST be in the range between the values set for + "min_instances" and "max_instances" properties. + type: integer + constraints: + - greater_or_equal: 0 + required: false + targets: + - tosca.nodes.Root http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/profiles/tosca-simple-1.0/capabilities.yaml ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/profiles/tosca-simple-1.0/capabilities.yaml b/extensions/aria_extension_tosca/profiles/tosca-simple-1.0/capabilities.yaml index 30abe10..b705d47 100644 --- a/extensions/aria_extension_tosca/profiles/tosca-simple-1.0/capabilities.yaml +++ b/extensions/aria_extension_tosca/profiles/tosca-simple-1.0/capabilities.yaml @@ -132,6 +132,7 @@ capability_types: specification: tosca-simple-1.0 specification_section: 5.4.10 specification_url: 'http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#DEFN_TYPE_CAPABILITIES_SCALABLE' + role: scaling description: >- This is the default TOSCA type that should be used to express a scalability capability for a node. derived_from: tosca.capabilities.Root http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/assignments.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/assignments.py b/extensions/aria_extension_tosca/simple_v1_0/assignments.py index 0590527..2cfc9e5 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/assignments.py +++ b/extensions/aria_extension_tosca/simple_v1_0/assignments.py @@ -32,6 +32,7 @@ from .presentation.field_validators import (node_template_or_type_validator, from .presentation.types import (convert_shorthand_to_full_type_name, get_type_by_full_or_shorthand_name) + @implements_specification('3.5.9', 'tosca-simple-1.0') class PropertyAssignment(AsIsPresentation): """ @@ -43,6 +44,7 @@ class PropertyAssignment(AsIsPresentation): #DEFN_ELEMENT_PROPERTY_VALUE_ASSIGNMENT>`__ """ + @short_form_field('implementation') @has_fields @implements_specification('3.5.13-2', 'tosca-simple-1.0') @@ -103,6 +105,7 @@ class OperationAssignment(ExtensiblePresentation): extensions.update(self._extensions) return extensions + @allow_unknown_fields @has_fields @implements_specification('3.5.14-2', 'tosca-simple-1.0') @@ -154,6 +157,7 @@ class InterfaceAssignment(ExtensiblePresentation): for operation in self.operations.itervalues(): # pylint: disable=no-member operation._validate(context) + @short_form_field('type') @has_fields class RelationshipAssignment(ExtensiblePresentation): @@ -202,6 +206,7 @@ class RelationshipAssignment(ExtensiblePresentation): return the_type, 'relationship_type' return None, None + @short_form_field('node') @has_fields @implements_specification('3.7.2', 'tosca-simple-1.0') @@ -301,6 +306,7 @@ class RequirementAssignment(ExtensiblePresentation): return None, None + @implements_specification('3.5.11', 'tosca-simple-1.0') class AttributeAssignment(AsIsPresentation): """ @@ -312,6 +318,7 @@ class AttributeAssignment(AsIsPresentation): #DEFN_ELEMENT_ATTRIBUTE_VALUE_ASSIGNMENT>`__ """ + @has_fields @implements_specification('3.7.1', 'tosca-simple-1.0') class CapabilityAssignment(ExtensiblePresentation): @@ -354,6 +361,7 @@ class CapabilityAssignment(ExtensiblePresentation): return capability_definition._get_type(context) \ if capability_definition is not None else None + @has_fields @implements_specification('3.5.6', 'tosca-simple-1.0') class ArtifactAssignment(ExtensiblePresentation): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/data_types.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/data_types.py b/extensions/aria_extension_tosca/simple_v1_0/data_types.py index c385f78..513b517 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/data_types.py +++ b/extensions/aria_extension_tosca/simple_v1_0/data_types.py @@ -14,14 +14,14 @@ # limitations under the License. import re -from datetime import datetime, tzinfo, timedelta +from datetime import (datetime, tzinfo, timedelta) try: from functools import total_ordering except ImportError: from total_ordering import total_ordering from aria.parser import implements_specification -from aria.utils.collections import StrictDict, OrderedDict +from aria.utils.collections import (StrictDict, OrderedDict) from aria.utils.formatting import safe_repr from .modeling.data_types import (coerce_to_data_type_class, report_issue_for_bad_format, @@ -48,8 +48,10 @@ class Timezone(tzinfo): _ZERO = timedelta(0) + UTC = Timezone() + @total_ordering @implements_specification('timestamp', 'yaml-1.1') class Timestamp(object): @@ -145,6 +147,7 @@ class Timestamp(object): def _fraction_as_str(the_datetime): return '{0:g}'.format(the_datetime.microsecond / 1000000.0).lstrip('0') + @total_ordering @implements_specification('3.2.2', 'tosca-simple-1.0') class Version(object): @@ -229,6 +232,7 @@ class Version(object): return True return False + @implements_specification('3.2.3', 'tosca-simple-1.0') class Range(object): """ @@ -276,6 +280,7 @@ class Range(object): def as_raw(self): return list(self.value) + @implements_specification('3.2.4', 'tosca-simple-1.0') class List(list): """ @@ -309,6 +314,7 @@ class List(list): def as_raw(self): return list(self) + @implements_specification('3.2.5', 'tosca-simple-1.0') class Map(StrictDict): """ @@ -348,6 +354,7 @@ class Map(StrictDict): def as_raw(self): return OrderedDict(self) + @total_ordering @implements_specification('3.2.6', 'tosca-simple-1.0') class Scalar(object): @@ -416,6 +423,7 @@ class Scalar(object): value = self.TYPE(scalar) # pylint: disable=no-member return self.value < value + @implements_specification('3.2.6.4', 'tosca-simple-1.0') class ScalarSize(Scalar): """ @@ -444,6 +452,7 @@ class ScalarSize(Scalar): TYPE = int UNIT = 'bytes' + @implements_specification('3.2.6.5', 'tosca-simple-1.0') class ScalarTime(Scalar): """ @@ -469,6 +478,7 @@ class ScalarTime(Scalar): TYPE = float UNIT = 'seconds' + @implements_specification('3.2.6.6', 'tosca-simple-1.0') class ScalarFrequency(Scalar): """ @@ -491,6 +501,7 @@ class ScalarFrequency(Scalar): TYPE = float UNIT = 'Hz' + # # The following are hooked in the YAML as 'coerce_value' extensions # @@ -499,10 +510,12 @@ def coerce_timestamp(context, presentation, the_type, entry_schema, constraints, return coerce_to_data_type_class(context, presentation, Timestamp, entry_schema, constraints, value, aspect) + def coerce_version(context, presentation, the_type, entry_schema, constraints, value, aspect): # pylint: disable=unused-argument return coerce_to_data_type_class(context, presentation, Version, entry_schema, constraints, value, aspect) + def coerce_range(context, presentation, the_type, entry_schema, constraints, value, aspect): if aspect == 'in_range': # When we're in a "in_range" constraint, the values are *not* themselves ranges, but numbers @@ -516,24 +529,29 @@ def coerce_range(context, presentation, the_type, entry_schema, constraints, val return coerce_to_data_type_class(context, presentation, Range, entry_schema, constraints, value, aspect) + def coerce_list(context, presentation, the_type, entry_schema, constraints, value, aspect): # pylint: disable=unused-argument return coerce_to_data_type_class(context, presentation, List, entry_schema, constraints, value, aspect) + def coerce_map_value(context, presentation, the_type, entry_schema, constraints, value, aspect): # pylint: disable=unused-argument return coerce_to_data_type_class(context, presentation, Map, entry_schema, constraints, value, aspect) + def coerce_scalar_unit_size(context, presentation, the_type, entry_schema, constraints, value, # pylint: disable=unused-argument aspect): return coerce_to_data_type_class(context, presentation, ScalarSize, entry_schema, constraints, value, aspect) + def coerce_scalar_unit_time(context, presentation, the_type, entry_schema, constraints, value, # pylint: disable=unused-argument aspect): return coerce_to_data_type_class(context, presentation, ScalarTime, entry_schema, constraints, value, aspect) + def coerce_scalar_unit_frequency(context, presentation, the_type, entry_schema, constraints, value, # pylint: disable=unused-argument aspect): return coerce_to_data_type_class(context, presentation, ScalarFrequency, entry_schema, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/definitions.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/definitions.py b/extensions/aria_extension_tosca/simple_v1_0/definitions.py index 1bd0366..eaa1ac9 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/definitions.py +++ b/extensions/aria_extension_tosca/simple_v1_0/definitions.py @@ -34,6 +34,7 @@ from .modeling.data_types import get_data_type, get_property_constraints from .modeling.interfaces import (get_and_override_input_definitions_from_type, get_and_override_operation_definitions_from_type) + @has_fields @implements_specification('3.5.8', 'tosca-simple-1.0') class PropertyDefinition(ExtensiblePresentation): @@ -120,6 +121,7 @@ class PropertyDefinition(ExtensiblePresentation): def _get_constraints(self, context): return get_property_constraints(context, self) + @has_fields @implements_specification('3.5.10', 'tosca-simple-1.0') class AttributeDefinition(ExtensiblePresentation): @@ -189,6 +191,7 @@ class AttributeDefinition(ExtensiblePresentation): def _get_type(self, context): return get_data_type(context, self, 'type') + @has_fields @implements_specification('3.5.12', 'tosca-simple-1.0') class ParameterDefinition(PropertyDefinition): @@ -223,6 +226,7 @@ class ParameterDefinition(PropertyDefinition): as the result from the evaluation of an expression or a function. """ + @short_form_field('implementation') @has_fields @implements_specification('3.5.13-1', 'tosca-simple-1.0') @@ -264,6 +268,7 @@ class OperationDefinition(ExtensiblePresentation): :type: {:obj:`basestring`: :class:`PropertyDefinition`} """ + @allow_unknown_fields @has_fields @implements_specification('3.5.14-1', 'tosca-simple-1.0') @@ -322,6 +327,7 @@ class InterfaceDefinition(ExtensiblePresentation): for operation in self.operations.itervalues(): # pylint: disable=no-member operation._validate(context) + @short_form_field('type') @has_fields class RelationshipDefinition(ExtensiblePresentation): @@ -354,6 +360,7 @@ class RelationshipDefinition(ExtensiblePresentation): def _get_type(self, context): return get_type_by_full_or_shorthand_name(context, self.type, 'relationship_types') + @short_form_field('capability') @has_fields @implements_specification('3.6.2', 'tosca-simple-1.0') @@ -420,6 +427,7 @@ class RequirementDefinition(ExtensiblePresentation): def _get_node_type(self, context): return context.presentation.get_from_dict('service_template', 'node_types', self.node) + @short_form_field('type') @has_fields @implements_specification('3.6.1', 'tosca-simple-1.0') http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/filters.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/filters.py b/extensions/aria_extension_tosca/simple_v1_0/filters.py index 6db140d..95d84b2 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/filters.py +++ b/extensions/aria_extension_tosca/simple_v1_0/filters.py @@ -22,6 +22,7 @@ from .presentation.extensible import ExtensiblePresentation from .presentation.field_validators import (node_filter_properties_validator, node_filter_capabilities_validator) + @has_fields class CapabilityFilter(ExtensiblePresentation): """ @@ -48,6 +49,7 @@ class CapabilityFilter(ExtensiblePresentation): return None + @has_fields @implements_specification('3.5.4', 'tosca-simple-1.0') class NodeFilter(ExtensiblePresentation): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/misc.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/misc.py b/extensions/aria_extension_tosca/simple_v1_0/misc.py index f4d43ac..fb86157 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/misc.py +++ b/extensions/aria_extension_tosca/simple_v1_0/misc.py @@ -36,6 +36,7 @@ from .presentation.field_validators import (constraint_clause_field_validator, from .presentation.types import (convert_shorthand_to_full_type_name, get_type_by_full_or_shorthand_name) + @implements_specification('3.5.1', 'tosca-simple-1.0') class Description(AsIsPresentation): """ @@ -53,6 +54,7 @@ class Description(AsIsPresentation): value = as_raw(self.value) puts(context.style.meta(value)) + @allow_unknown_fields @has_fields @implements_specification('3.9.3.2', 'tosca-simple-1.0') @@ -91,6 +93,7 @@ class MetaData(ExtensiblePresentation): :type: dict """ + @short_form_field('url') @has_fields @implements_specification('3.5.5', 'tosca-simple-1.0') @@ -132,6 +135,7 @@ class Repository(ExtensiblePresentation): def _get_credential(self, context): return get_data_type_value(context, self, 'credential', 'tosca.datatypes.Credential') + @short_form_field('file') @has_fields @implements_specification('3.5.7', 'tosca-simple-1.0') @@ -182,6 +186,7 @@ class Import(ExtensiblePresentation): :type: :obj:`basestring` """ + @has_fields @implements_specification('3.5.2-1', 'tosca-simple-1.0') class ConstraintClause(ExtensiblePresentation): @@ -294,6 +299,7 @@ class ConstraintClause(ExtensiblePresentation): def _apply_to_value(self, context, presentation, value): return apply_constraint_to_value(context, presentation, self, value) + @short_form_field('type') @has_fields class EntrySchema(ExtensiblePresentation): @@ -329,6 +335,7 @@ class EntrySchema(ExtensiblePresentation): def _get_constraints(self, context): return get_property_constraints(context, self) + @short_form_field('primary') @has_fields class OperationImplementation(ExtensiblePresentation): @@ -355,6 +362,7 @@ class OperationImplementation(ExtensiblePresentation): :type: [:obj:`basestring`] """ + class SubstitutionMappingsRequirement(AsIsPresentation): """ Substitution mapping for requirement. @@ -374,6 +382,7 @@ class SubstitutionMappingsRequirement(AsIsPresentation): super(SubstitutionMappingsRequirement, self)._validate(context) validate_subtitution_mappings_requirement(context, self) + class SubstitutionMappingsCapability(AsIsPresentation): """ Substitution mapping for capability. @@ -393,6 +402,7 @@ class SubstitutionMappingsCapability(AsIsPresentation): super(SubstitutionMappingsCapability, self)._validate(context) validate_subtitution_mappings_capability(context, self) + @has_fields @implements_specification('2.10', 'tosca-simple-1.0') class SubstitutionMappings(ExtensiblePresentation): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/modeling/__init__.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/__init__.py b/extensions/aria_extension_tosca/simple_v1_0/modeling/__init__.py index 957dc7b..4d8e1db 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/modeling/__init__.py +++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/__init__.py @@ -168,9 +168,6 @@ def create_node_template_model(context, service_template, node_template): model = NodeTemplate(name=node_template._name, type=node_type) - model.default_instances = 1 - model.min_instances = 0 - if node_template.description: model.description = node_template.description.value http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/modeling/artifacts.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/artifacts.py b/extensions/aria_extension_tosca/simple_v1_0/modeling/artifacts.py index dd9eeb4..b45615a 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/modeling/artifacts.py +++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/artifacts.py @@ -21,6 +21,8 @@ from aria.utils.collections import OrderedDict # def get_inherited_artifact_definitions(context, presentation, for_presentation=None): + if for_presentation is None: + for_presentation = presentation if hasattr(presentation, '_get_type'): # In NodeTemplate @@ -30,7 +32,7 @@ def get_inherited_artifact_definitions(context, presentation, for_presentation=N parent = presentation._get_parent(context) # Get artifact definitions from parent - artifacts = get_inherited_artifact_definitions(context, parent, for_presentation=presentation) \ + artifacts = get_inherited_artifact_definitions(context, parent, for_presentation) \ if parent is not None else OrderedDict() # Add/override our artifact definitions http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py b/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py index d75e723..760541a 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py +++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py @@ -52,12 +52,13 @@ def get_inherited_capability_definitions(context, presentation, for_presentation Allows overriding all aspects of parent capability properties except data type. """ + if for_presentation is None: + for_presentation = presentation + # Get capability definitions from parent parent = presentation._get_parent(context) - capability_definitions = get_inherited_capability_definitions(context, parent, - for_presentation=presentation) \ - if parent is not None \ - else OrderedDict() + capability_definitions = get_inherited_capability_definitions( + context, parent, for_presentation) if parent is not None else OrderedDict() # Add/merge our capability definitions our_capability_definitions = presentation.capabilities @@ -71,13 +72,12 @@ def get_inherited_capability_definitions(context, presentation, for_presentation type2 = our_capability_definition.type if type1 != type2: context.validation.report( - 'capability definition changes type from "%s" to "%s" in "%s"' - % (type1, type2, presentation._fullname), + 'capability definition changes type from "{0}" to "{1}" in "{2}"' + .format(type1, type2, presentation._fullname), locator=our_capability_definition._locator, level=Issue.BETWEEN_TYPES) - # Already cloned? - #capability_definition = capability_definition._clone(for_presentation) - #capability_definitions[capability_name] = capability_definition + merge_capability_definition(context, presentation, capability_definition, + our_capability_definition) else: capability_definition = our_capability_definition._clone(for_presentation) if isinstance(capability_definition._raw, basestring): @@ -133,12 +133,14 @@ def get_template_capabilities(context, presentation): values = get_assigned_and_defined_parameter_values(context, our_capability_assignment, 'property') + if values: capability_assignment._raw['properties'] = values + capability_assignment._reset_method_cache() else: context.validation.report( - 'capability "%s" not declared at node type "%s" in "%s"' - % (capability_name, presentation.type, presentation._fullname), + 'capability "{0}" not declared at node type "{1}" in "{2}"' + .format(capability_name, presentation.type, presentation._fullname), locator=our_capability_assignment._locator, level=Issue.BETWEEN_TYPES) return capability_assignments @@ -162,6 +164,30 @@ def convert_capability_from_definition_to_assignment(context, presentation, cont return CapabilityAssignment(name=presentation._name, raw=raw, container=container) +def merge_capability_definition(context, presentation, capability_definition, + from_capability_definition): + raw_properties = OrderedDict() + + # Merge properties from type + from_property_defintions = from_capability_definition.properties + merge_raw_parameter_definitions(context, presentation, raw_properties, from_property_defintions, + 'properties') + + # Merge our properties + merge_raw_parameter_definitions(context, presentation, raw_properties, + capability_definition.properties, 'properties') + + if raw_properties: + capability_definition._raw['properties'] = raw_properties + capability_definition._reset_method_cache() + + # Merge occurrences + occurrences = from_capability_definition._raw.get('occurrences') + if (occurrences is not None) and (capability_definition._raw.get('occurrences') is None): + capability_definition._raw['occurrences'] = \ + deepcopy_with_locators(occurrences) + + def merge_capability_definition_from_type(context, presentation, capability_definition): raw_properties = OrderedDict() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/modeling/interfaces.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/interfaces.py b/extensions/aria_extension_tosca/simple_v1_0/modeling/interfaces.py index e04ac4a..d5f447c 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/modeling/interfaces.py +++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/interfaces.py @@ -113,9 +113,12 @@ def get_inherited_interface_definitions(context, presentation, type_name, for_pr types. """ + if for_presentation is None: + for_presentation = presentation + # Get interfaces from parent parent = presentation._get_parent(context) - interfaces = get_inherited_interface_definitions(context, parent, type_name, presentation) \ + interfaces = get_inherited_interface_definitions(context, parent, type_name, for_presentation) \ if parent is not None else OrderedDict() # Add/merge interfaces from their types @@ -123,8 +126,7 @@ def get_inherited_interface_definitions(context, presentation, type_name, for_pr # Add/merge our interfaces our_interfaces = presentation.interfaces - merge_interface_definitions(context, interfaces, our_interfaces, presentation, - for_presentation=for_presentation) + merge_interface_definitions(context, interfaces, our_interfaces, presentation, for_presentation) return interfaces http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/modeling/parameters.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/parameters.py b/extensions/aria_extension_tosca/simple_v1_0/modeling/parameters.py index c910956..87c1a3b 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/modeling/parameters.py +++ b/extensions/aria_extension_tosca/simple_v1_0/modeling/parameters.py @@ -33,11 +33,14 @@ def get_inherited_parameter_definitions(context, presentation, field_name, for_p Allows overriding all aspects of parent properties except data type. """ + if for_presentation is None: + for_presentation = presentation + # Get definitions from parent # If we inherit from a primitive, it does not have a parent: parent = presentation._get_parent(context) if hasattr(presentation, '_get_parent') else None definitions = get_inherited_parameter_definitions(context, parent, field_name, - for_presentation=presentation) \ + for_presentation) \ if parent is not None else OrderedDict() # Add/merge our definitions http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/presentation/extensible.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/presentation/extensible.py b/extensions/aria_extension_tosca/simple_v1_0/presentation/extensible.py index 63bc02f..0e3c94d 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/presentation/extensible.py +++ b/extensions/aria_extension_tosca/simple_v1_0/presentation/extensible.py @@ -16,6 +16,7 @@ from aria.utils.caching import cachedmethod from aria.parser.presentation import (Presentation, has_fields, primitive_dict_field) + @has_fields class ExtensiblePresentation(Presentation): """ http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py index be80702..fc9a9bc 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py +++ b/extensions/aria_extension_tosca/simple_v1_0/presentation/field_validators.py @@ -24,6 +24,7 @@ from ..modeling.data_types import (get_primitive_data_type, get_data_type_name, get_container_data_type) from .types import get_type_by_full_or_shorthand_name, convert_shorthand_to_full_type_name + # # NodeTemplate, RelationshipTemplate # @@ -56,6 +57,7 @@ def copy_validator(template_type_name, templates_dict_name): return validator_fn + # # PropertyDefinition, AttributeDefinition, ParameterDefinition, EntrySchema # @@ -97,6 +99,7 @@ def data_type_validator(type_name='data type'): return validator + # # PropertyDefinition, AttributeDefinition # @@ -141,6 +144,7 @@ def entry_schema_validator(field, presentation, context): % (get_data_type_name(the_type), presentation._container._fullname), locator=presentation._locator, level=Issue.BETWEEN_TYPES) + def data_value_validator(field, presentation, context): """ Makes sure that the field contains a valid value according to data type and constraints. @@ -160,6 +164,7 @@ def data_value_validator(field, presentation, context): if hasattr(presentation, '_get_constraints') else None coerce_value(context, presentation, the_type, entry_schema, constraints, value, field.name) + # # DataType # @@ -168,6 +173,7 @@ _data_type_validator = data_type_validator() _data_type_derived_from_validator = derived_from_validator(convert_shorthand_to_full_type_name, 'data_types') + def data_type_derived_from_validator(field, presentation, context): """ Makes sure that the field refers to a valid parent data type (complex or primitive). @@ -181,6 +187,7 @@ def data_type_derived_from_validator(field, presentation, context): # hierarchy) _data_type_derived_from_validator(field, presentation, context) + def data_type_constraints_validator(field, presentation, context): """ Makes sure that we do not have constraints if we are a complex type (with no primitive @@ -197,6 +204,7 @@ def data_type_constraints_validator(field, presentation, context): % presentation._fullname, locator=presentation._get_child_locator(field.name), level=Issue.BETWEEN_TYPES) + def data_type_properties_validator(field, presentation, context): """ Makes sure that we do not have properties if we have a primitive ancestor. @@ -215,6 +223,7 @@ def data_type_properties_validator(field, presentation, context): % presentation._fullname, locator=presentation._get_child_locator(field.name), level=Issue.BETWEEN_TYPES) + # # ConstraintClause # @@ -235,6 +244,7 @@ def constraint_clause_field_validator(field, presentation, context): if hasattr(the_type, '_get_constraints') else None coerce_value(context, presentation, the_type, None, constraints, value, field.name) + def constraint_clause_in_range_validator(field, presentation, context): """ Makes sure that the value is a list with exactly two elements, that both lower bound contains a @@ -274,6 +284,7 @@ def constraint_clause_in_range_validator(field, presentation, context): % (field.name, presentation._fullname), locator=presentation._get_child_locator(field.name), level=Issue.FIELD) + def constraint_clause_valid_values_validator(field, presentation, context): """ Makes sure that the value is a list of valid values for the container type. @@ -290,6 +301,7 @@ def constraint_clause_valid_values_validator(field, presentation, context): for value in values: coerce_value(context, presentation, the_type, None, None, value, field.name) + def constraint_clause_pattern_validator(field, presentation, context): """ Makes sure that the value is a valid regular expression. @@ -316,6 +328,7 @@ def constraint_clause_pattern_validator(field, presentation, context): % (field.name, presentation._fullname), locator=presentation._get_child_locator(field.name), level=Issue.FIELD, exception=e) + # # RequirementAssignment # @@ -340,6 +353,7 @@ def node_template_or_type_validator(field, presentation, context): report_issue_for_unknown_type(context, presentation, 'node template or node type', field.name) + def capability_definition_or_type_validator(field, presentation, context): """ Makes sure refers to either a capability assignment name in the node template referred to by the @@ -381,6 +395,7 @@ def capability_definition_or_type_validator(field, presentation, context): % (presentation._name, presentation._container._fullname, safe_repr(value)), locator=presentation._get_child_locator(field.name), level=Issue.BETWEEN_TYPES) + def node_filter_validator(field, presentation, context): """ Makes sure that the field has a value only if "node" refers to a node type. @@ -401,6 +416,7 @@ def node_filter_validator(field, presentation, context): % (presentation._fullname, presentation._container._fullname), locator=presentation._locator, level=Issue.BETWEEN_FIELDS) + # # RelationshipAssignment # @@ -426,6 +442,7 @@ def relationship_template_or_type_validator(field, presentation, context): report_issue_for_unknown_type(context, presentation, 'relationship template or relationship type', field.name) + # # PolicyType # @@ -449,6 +466,7 @@ def list_node_type_or_group_type_validator(field, presentation, context): report_issue_for_unknown_type(context, presentation, 'node type or group type', field.name, value) + # # PolicyTemplate # @@ -506,6 +524,7 @@ def policy_targets_validator(field, presentation, context): % (presentation._name, safe_repr(value)), locator=presentation._locator, level=Issue.BETWEEN_TYPES) + # # NodeFilter # @@ -532,6 +551,7 @@ def node_filter_properties_validator(field, presentation, context): % (node_type._name, name), locator=presentation._locator, level=Issue.BETWEEN_TYPES) + def node_filter_capabilities_validator(field, presentation, context): """ Makes sure that the field's elements refer to defined capabilities and properties in the target http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/presentation/types.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/presentation/types.py b/extensions/aria_extension_tosca/simple_v1_0/presentation/types.py index 610e4a0..2f30e0f 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/presentation/types.py +++ b/extensions/aria_extension_tosca/simple_v1_0/presentation/types.py @@ -13,6 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the License. + def convert_shorthand_to_full_type_name(context, name, types_dict): # pylint: disable=unused-argument """ Converts a shorthand type name to its full type name, or else returns it unchanged. @@ -31,6 +32,7 @@ def convert_shorthand_to_full_type_name(context, name, types_dict): # pylint: di return full_name return name + def get_type_by_full_or_shorthand_name(context, name, *types_dict_names): """ Gets a type either by its full name or its shorthand name. http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/extensions/aria_extension_tosca/simple_v1_0/presenter.py ---------------------------------------------------------------------- diff --git a/extensions/aria_extension_tosca/simple_v1_0/presenter.py b/extensions/aria_extension_tosca/simple_v1_0/presenter.py index 394e303..28c9f7b 100644 --- a/extensions/aria_extension_tosca/simple_v1_0/presenter.py +++ b/extensions/aria_extension_tosca/simple_v1_0/presenter.py @@ -13,7 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -from aria.utils.collections import FrozenList, EMPTY_READ_ONLY_LIST +from aria.utils.collections import (FrozenList, EMPTY_READ_ONLY_LIST) from aria.utils.caching import cachedmethod from aria.parser.presentation import Presenter @@ -22,6 +22,7 @@ from .modeling.functions import (Concat, Token, GetInput, GetProperty, GetAttrib GetOperationOutput, GetNodesOfType, GetArtifact) from .templates import ServiceTemplate + class ToscaSimplePresenter1_0(Presenter): # pylint: disable=invalid-name,abstract-method """ ARIA presenter for the `TOSCA Simple Profile v1.0 cos01 0 http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/tests/end2end/test_nodecellar.py ---------------------------------------------------------------------- diff --git a/tests/end2end/test_nodecellar.py b/tests/end2end/test_nodecellar.py index f02441f..e8cfa84 100644 --- a/tests/end2end/test_nodecellar.py +++ b/tests/end2end/test_nodecellar.py @@ -39,4 +39,4 @@ def _verify_deployed_service_in_storage(service_name, model_storage): service = service_templates[0].services[service_name] assert service.name == service_name assert len(service.executions) == 0 # dry executions leave no traces - assert len(service.nodes) == 10 + assert len(service.nodes) == 15 http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/tests/mock/models.py ---------------------------------------------------------------------- diff --git a/tests/mock/models.py b/tests/mock/models.py index 23a14bd..8a3b87e 100644 --- a/tests/mock/models.py +++ b/tests/mock/models.py @@ -133,10 +133,7 @@ def create_node_template(service_template, type=models.Type(variant='node', name='test_node_type'), capability_templates=None, requirement_templates=None, - interface_templates=None, - default_instances=1, - min_instances=1, - max_instances=1): + interface_templates=None): capability_templates = capability_templates or {} requirement_templates = requirement_templates or [] interface_templates = interface_templates or {} @@ -146,9 +143,6 @@ def create_node_template(service_template, capability_templates=capability_templates, requirement_templates=requirement_templates, interface_templates=interface_templates, - default_instances=default_instances, - min_instances=min_instances, - max_instances=max_instances, service_template=service_template) service_template.node_templates[node_template.name] = node_template http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ee9c08a0/tests/modeling/test_mixins.py ---------------------------------------------------------------------- diff --git a/tests/modeling/test_mixins.py b/tests/modeling/test_mixins.py index 2c91a4b..2d94d7c 100644 --- a/tests/modeling/test_mixins.py +++ b/tests/modeling/test_mixins.py @@ -112,9 +112,6 @@ def test_relationship_model_ordering(context): new_node_template = modeling.models.NodeTemplate( name='new_node_template', type=source_node.type, - default_instances=1, - min_instances=1, - max_instances=1, service_template=service.service_template )