Mailing-List: contact dev-help@brooklyn.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@brooklyn.apache.org
From: neykov <git@git.apache.org>
To: dev@brooklyn.apache.org
Reply-To: dev@brooklyn.apache.org
References: <git-pr-529-brooklyn-server@git.apache.org>
In-Reply-To: <git-pr-529-brooklyn-server@git.apache.org>
Subject: [GitHub] brooklyn-server pull request #529: LocationNetworkInfoCustomizer
Content-Type: text/plain
Message-Id: <20170126150428.36578F403D@git1-us-west.apache.org>
Date: Thu, 26 Jan 2017 15:04:28 +0000 (UTC)
archived-at: Thu, 26 Jan 2017 15:04:31 -0000

Github user neykov commented on a diff in the pull request:

    https://github.com/apache/brooklyn-server/pull/529#discussion_r97972489
  
    --- Diff: locations/jclouds/src/main/java/org/apache/brooklyn/location/jclouds/BasicLocationNetworkInfoCustomizer.java ---
    @@ -0,0 +1,472 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing,
    + * software distributed under the License is distributed on an
    + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    + * KIND, either express or implied.  See the License for the
    + * specific language governing permissions and limitations
    + * under the License.
    + */
    +
    +package org.apache.brooklyn.location.jclouds;
    +
    +import java.util.Iterator;
    +import java.util.Map;
    +
    +import org.apache.brooklyn.api.entity.Entity;
    +import org.apache.brooklyn.api.entity.EntityInitializer;
    +import org.apache.brooklyn.api.entity.EntityLocal;
    +import org.apache.brooklyn.api.sensor.AttributeSensor;
    +import org.apache.brooklyn.config.ConfigKey;
    +import org.apache.brooklyn.core.config.ConfigKeys;
    +import org.apache.brooklyn.core.entity.Attributes;
    +import org.apache.brooklyn.core.entity.BrooklynConfigKeys;
    +import org.apache.brooklyn.core.location.LocationConfigKeys;
    +import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
    +import org.apache.brooklyn.core.sensor.Sensors;
    +import org.apache.brooklyn.location.winrm.WinRmMachineLocation;
    +import org.apache.brooklyn.util.core.config.ConfigBag;
    +import org.apache.brooklyn.util.core.task.Tasks;
    +import org.apache.brooklyn.util.exceptions.Exceptions;
    +import org.apache.brooklyn.util.guava.Maybe;
    +import org.apache.brooklyn.util.net.Networking;
    +import org.apache.brooklyn.util.time.Duration;
    +import org.jclouds.compute.domain.NodeMetadata;
    +import org.jclouds.domain.LoginCredentials;
    +import org.slf4j.Logger;
    +import org.slf4j.LoggerFactory;
    +
    +import com.google.common.annotations.Beta;
    +import com.google.common.base.MoreObjects;
    +import com.google.common.base.Optional;
    +import com.google.common.base.Predicate;
    +import com.google.common.base.Stopwatch;
    +import com.google.common.base.Supplier;
    +import com.google.common.base.Suppliers;
    +import com.google.common.collect.ImmutableList;
    +import com.google.common.collect.Iterables;
    +import com.google.common.net.HostAndPort;
    +
    +/**
    + * BasicLocationNetworkInfoCustomizer provides the default implementation of
    + * {@link LocationNetworkInfoCustomizer}. It exposes options to have JcloudsLocation
    + * prefer to contact VMs on private addresses and can be injected on a
    + * per-entity basis. For example:
    + * <pre>
    + * services:
    + * - type: server
    + *   location: the-same-private-network-as-brooklyn
    + *   brooklyn.initializers:
    + *   - type: org.apache.brooklyn.location.jclouds.BasicLocationNetworkInfoCustomizer
    + *     brooklyn.config:
    + *       mode: ONLY_PRIVATE
    + * - type: server
    + *   location: another-cloud
    + *   # implicit use of PREFER_PUBLIC.
    + * </pre>
    + * Would result in the first entity being managed on the instance's private address (and deployment
    + * failing if this was not possible) and the second being managed on its public address. Graceful
    + * fallback is possible by replacing ONLY_PRIVATE with PREFER_PRIVATE. There are PUBLIC variants of
    + * each of these.
    + * <p>
    + * BasicLocationNetworkInfoCustomizer is the default location network info customizer used by
    + * {@link JcloudsLocation} when {@link JcloudsLocationConfig#LOCATION_NETWORK_INFO_CUSTOMIZER}
    + * is unset.
    + * <p>
    + * When used as an {@link EntityInitializer} the instance inserts itself into the entity's
    + * provisioning properties under the {@link JcloudsLocationConfig#LOCATION_NETWORK_INFO_CUSTOMIZER}
    + * subkey.
    + * <p>
    + * This class is annotated @Beta and is likely to change in the future.
    + */
    +@Beta
    +public class BasicLocationNetworkInfoCustomizer extends BasicJcloudsLocationCustomizer implements LocationNetworkInfoCustomizer {
    +
    +    private static final Logger LOG = LoggerFactory.getLogger(BasicLocationNetworkInfoCustomizer.class);
    +
    +    public enum NetworkMode {
    +        /**
    +         * Check each node's {@link NodeMetadata#getPublicAddresses() public addresses}
    +         * for reachability before its {@link NodeMetadata#getPrivateAddresses() private addresses}.
    +         */
    +        PREFER_PUBLIC,
    +        /**
    +         * Check each node's {@link NodeMetadata#getPrivateAddresses() private addresses}
    +         * for reachability before its {@link NodeMetadata#getPublicAddresses() public addresses}.
    +         */
    +        PREFER_PRIVATE,
    +        /**
    +         * Check only a node's {@link NodeMetadata#getPublicAddresses() public addresses} for reachability.
    +         */
    +        ONLY_PUBLIC,
    +        /**
    +         * Check only a node's {@link NodeMetadata#getPrivateAddresses()}  private addresses} for reachability.
    +         */
    +        ONLY_PRIVATE
    +    }
    +
    +    public static final ConfigKey<NetworkMode> MODE = ConfigKeys.newConfigKey(NetworkMode.class,
    +            "mode", "Operation mode", NetworkMode.PREFER_PUBLIC);
    +
    +    @Beta
    +    public static final ConfigKey<Boolean> CHECK_CREDENTIALS = ConfigKeys.newBooleanConfigKey(
    +            "checkCredentials",
    +            "Indicates that credentials should be tested when determining endpoint reachability.",
    +            Boolean.TRUE);
    +
    +    public static final ConfigKey<Boolean> PUBLISH_NETWORKS = ConfigKeys.newBooleanConfigKey(
    +            "publishNetworks",
    +            "Indicates that the customizer should publish addresses as sensors on each entity",
    +            Boolean.TRUE);
    +
    +    // --------------------------------------------------------------------------------------
    +
    +    public BasicLocationNetworkInfoCustomizer() {
    +        super();
    +    }
    +
    +    public BasicLocationNetworkInfoCustomizer(Map<?, ?> params) {
    +        super(params);
    +    }
    +
    +    public BasicLocationNetworkInfoCustomizer(final ConfigBag params) {
    +        super(params);
    +    }
    +
    +    // --------------------------------------------------------------------------------------
    +
    +    /**
    +     * Overrides the behaviour of {@link BasicJcloudsLocationCustomizer#apply(EntityLocal)} to set
    +     * the instance as the value of {@link JcloudsLocationConfig#LOCATION_NETWORK_INFO_CUSTOMIZER},
    +     * rather than in its provisioning properties.
    +     */
    +    @Override
    +    public void apply(EntityLocal entity) {
    +        ConfigKey<Object> subkey = BrooklynConfigKeys.PROVISIONING_PROPERTIES.subKey(JcloudsLocationConfig.LOCATION_NETWORK_INFO_CUSTOMIZER.getName());
    +        entity.config().set(subkey, this);
    +        LOG.debug("{} set itself as the location network info customizer on {}", this, entity);
    +    }
    +
    +    // --------------------------------------------------------------------------------------
    +
    +    /**
    +     * Combines the given resolve options with the customiser's configuration to determine the
    +     * best address and credential pair for management. In particular, if the resolve options
    +     * allow it will check that the credential is actually valid for the address.
    +     */
    +    @Override
    +    public ManagementAddressResolveResult resolve(
    +            JcloudsLocation location, NodeMetadata node, ConfigBag config, ManagementAddressResolveOptions options) {
    +        LOG.debug("{} resolving management parameters for {}, node={}, config={}, options={}",
    +                new Object[]{this, location, node, config, options});
    +        Stopwatch timer = Stopwatch.createStarted();
    +        // Should only be null in tests.
    +        final Entity contextEntity = getContextEntity(config);
    +        if (shouldPublishNetworks() && options.publishNetworkSensors() && contextEntity != null) {
    +            publishNetworks(node, contextEntity);
    +        }
    +        HostAndPort hapChoice = null;
    +        LoginCredentials credChoice = null;
    +
    +        Iterable<HostAndPort> managementCandidates = getManagementCandidates(location, node, config, options);
    +        Iterable<LoginCredentials> credentialCandidates = getCredentialCandidates(location, node, options, config);
    +
    +        // Try each pair of address and credential until one succeeds.
    +        if (options.expectReachable() && options.pollForFirstReachableAddress() && shouldCheckCredentials()) {
    +            for (HostAndPort hap : managementCandidates) {
    +                for (LoginCredentials cred : credentialCandidates) {
    +                    LOG.trace("Testing host={} with credential={}", hap, cred);
    +                    if (checkCredential(location, hap, cred, config, options.isWindows())) {
    +                        hapChoice = hap;
    +                        credChoice = cred;
    +                        break;
    +                    }
    +                }
    +                if (hapChoice != null) break;
    +            }
    +        }
    +
    +        if (hapChoice == null) {
    +            LOG.trace("Choosing first management candidate given node={} and mode={}", node, getMode());
    +            hapChoice = Iterables.getFirst(managementCandidates, null);
    +        }
    +        if (hapChoice == null) {
    +            LOG.trace("Choosing first address of node={} in mode={}", node, getMode());
    +            final Iterator<String> hit = getNodeAddressesWithMode(node).iterator();
    +            if (hit.hasNext()) HostAndPort.fromHost(hit.next());
    +        }
    +        if (hapChoice == null) {
    +            throw new IllegalStateException("Exhausted all options when determining address for " + location);
    +        }
    +
    +        if (credChoice == null) {
    +            credChoice = Iterables.getFirst(credentialCandidates, null);
    +            if (credChoice == null) {
    +                throw new IllegalStateException("Exhausted all options when determining credential for " + location);
    +            }
    +        }
    +
    +        if (contextEntity != null) {
    +            contextEntity.sensors().set(Attributes.ADDRESS, hapChoice.getHostText());
    +        }
    +        ManagementAddressResolveResult result = new ManagementAddressResolveResult(hapChoice, credChoice);
    +        LOG.debug("{} resolved management parameters for {} in {}: {}",
    +                new Object[]{this, location, Duration.of(timer), result});
    +        return result;
    +    }
    +
    +    private boolean shouldPublishNetworks() {
    +        return Boolean.TRUE.equals(config().get(PUBLISH_NETWORKS));
    +    }
    +
    +    // TODO: Separate this into second part?
    +    void publishNetworks(NodeMetadata node, Entity entity) {
    +        // todo hostnames?
    +        int i = 0;
    +        for (String address : node.getPrivateAddresses()) {
    +            final AttributeSensor<String> sensor = Sensors.newStringSensor("host.address.private." + i++);
    +            if (entity.sensors().get(sensor) == null) {
    +                entity.sensors().set(sensor, address);
    +            }
    +        }
    +        i = 0;
    +        for (String address : node.getPublicAddresses()) {
    +            final AttributeSensor<String> sensor = Sensors.newStringSensor("host.address.public." + i++);
    +            if (entity.sensors().get(sensor) == null) {
    +                entity.sensors().set(sensor, address);
    +            }
    +        }
    --- End diff --
    
    Do you have a particular usage of the sensors in mind? I'm wondering whether an array in a single sensor will be more useful. It can be processed by transformers and such. Having separate keys with unknown count makes any address > `host.address.xxx.0` useless.
    
    Is it possible that we have more than one hostname per machine? That's re the `todo hostnames` comment.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---