Mailing-List: contact dev-help@deltacloud.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@deltacloud.apache.org
Received-SPF: pass (athena.apache.org: domain of lutter@redhat.com designates
 209.132.183.28 as permitted sender)
Message-ID: <1360091982.3104.20.camel@melon.watzmann.net>
Subject: Re: making cimi tests pass on fgcp
From: David Lutterkort <lutter@redhat.com>
To: dev@deltacloud.apache.org
Date: Tue, 05 Feb 2013 11:19:42 -0800
In-Reply-To: <434A0ECB689CAF49A3A2321F30F2AB8324C1A44A@FALEX03.au.fjanz.com>
References: <434A0ECB689CAF49A3A2321F30F2AB8324AE6A0A@FALEX03.au.fjanz.com>
	 <1360017283.11380.8.camel@avon.watzmann.net>
	 <434A0ECB689CAF49A3A2321F30F2AB8324C1A44A@FALEX03.au.fjanz.com>
Organization: Red Hat Inc
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Tue, 2013-02-05 at 15:02 +1100, Koper, Dies wrote:
> So I prefer to apply Band-Aids for a little while longer, so we can at
> least keep testing the cimi front-end with fgcp easily as its coverage
> increases.
> I'd rather look at your solution as a plan B in case the problem will
> not be solved on the fgcp api side and more cimi clients appear.

The problem with applying bandaids to keep the tests working for FGCP is
that we are adapting to non-compliant behavior of the endpoint; and that
also affects the other frontends (DC , EC2)

> > There's actually a good case to be made to add task queue capabilities
> > to Deltacloud for other things, too, and we might get somebody else to
> 
> Yes, I did give it some thinking last month and it's basically a job
> engine we'd have to develop.
> So plan B may be to piggy back on the cimi jobs implementation.
> There is of course the issue that no clients (including DC tests) are
> written to work with that but at least it'd be cimi compliant :-p

This wouldn't have anything to do with CIMI jobs - it would internally
be built on a task queue, but that queue would be invisible to clients.
As an example, right now when a client requests an instance be created,
the driver just turns around and calls the FGCP API. To properly work
around this issue, the driver would throw that instance create request
into a task queue, from which a worker pulls jobs as FGCP is able to
process them. Of course, that means that for every 'GET instance' etc.
request the driver now has to look in the queue and the FGCP backend, as
instances the client thinks of as existing can either be only in the
task queue (and should be reported as pending) or they can be fully
created in FGCP.

> What a queuing system could do, is first try the operation regardless
> (as even if the queue is full, fgcp endpoint will do parameter
> validation) so that in case of invalid parameters or invalid state
> (e.g. trying to delete a machine that is not in the stopped state) the
> client can be notified directly.
> If the endpoint returns an error that the system is locked, add it to
> the queue. The queue periodically retries the operations.
> With machine and volume create operations, we need to return an id. We
> could use the system id + machine name, so at least the machine could
> be programmatically identified later (unless the client creates
> machines with the same name).
> With the public IP address... I suppose a (stateful) job mechanism is the only way?

All of these will require state, and as you point out, some measure of
id generation/translation; but yes, what you describe above is the
correct approach.

David