Return-Path: X-Original-To: apmail-airavata-dev-archive@www.apache.org Delivered-To: apmail-airavata-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 29B8A10CE7 for ; Mon, 13 Jan 2014 19:20:38 +0000 (UTC) Received: (qmail 26824 invoked by uid 500); 13 Jan 2014 17:52:16 -0000 Delivered-To: apmail-airavata-dev-archive@airavata.apache.org Received: (qmail 26726 invoked by uid 500); 13 Jan 2014 17:52:08 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 26497 invoked by uid 99); 13 Jan 2014 17:51:35 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Jan 2014 17:51:35 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of samindaw@gmail.com designates 209.85.217.169 as permitted sender) Received: from [209.85.217.169] (HELO mail-lb0-f169.google.com) (209.85.217.169) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Jan 2014 17:51:30 +0000 Received: by mail-lb0-f169.google.com with SMTP id q8so2675034lbi.28 for ; Mon, 13 Jan 2014 09:51:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=I3+H6DP/Xc0MyreXS1yOMdcZwD4qjeMOWfk6s3E27uE=; b=GUdlZM9pdDg/kTPlsunpTshzsZRNQ0Uv/5Y9moUdEtd7bxE2C/KB8cuCxQ/BEAn9+T iT11pVoGZt/AlSo/gLDVcMUm8Po4uTceznaUhhnJBMVwYQYB5HU9MhHGuYWRN+BXOWUK sROjEDu0m621+OOQ1OlRYBS68igcv4qjKpvoTgC0qmPimzmr+RThvAzWL8N/H/Hsx5EJ jfrdcj0XshqUHNT13MLFjZQaYdFg9ZR4b2aMy4aiLfFhTe9jNW3nBgfXlv9+1UvyslWu LjFYNuXbsugkG234CYQXnMijvJbbRWxngx4ClFBCtfTIgVqPIxlZBU8tVYT+hTxViGcb IILQ== X-Received: by 10.112.63.104 with SMTP id f8mr260938lbs.79.1389635468974; Mon, 13 Jan 2014 09:51:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.161.100 with HTTP; Mon, 13 Jan 2014 09:50:48 -0800 (PST) In-Reply-To: <98D718CF-FAC7-46EE-BBA3-BBDAACC85F2D@apache.org> References: <98D718CF-FAC7-46EE-BBA3-BBDAACC85F2D@apache.org> From: Saminda Wijeratne Date: Mon, 13 Jan 2014 12:50:48 -0500 Message-ID: Subject: Re: Predefined parent working directory for jobs To: dev Content-Type: multipart/alternative; boundary=001a11c3bc4a7ede9604efddb86b X-Virus-Checked: Checked by ClamAV on apache.org --001a11c3bc4a7ede9604efddb86b Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi All, Thanks for all the responses. Lahiru, I will look at implementing the class *BaseDirectorySetupHandler. *= Is there a way which the gateway id and user can be obtained inside the handler? Suresh, this was also my initial thought. to use the applicationDeploymentDescriptor properties. While it is a convenient place for administrators to specify since it's documented in schema (so to speak), they have to do it for each application deployment and if the location changes they have to manually update all of them. But since I'm aiming for a demo at the moment this indeed is a good way to go as a short term solution. On Mon, Jan 13, 2014 at 11:14 AM, Suresh Marru wrote: > Saminda, Lahiru, > > Why not use the staticWorkingDirectory option within Application > Deployment Description [1]? The intention of this option was, within the > provider, if this is provided, it ignores the scratchWorkingDirectory and > always uses a static working directory location. > > Also, if I re-read Saminda=92s original request, is it not the normal GFa= c > behavior? I mean, within the specified directory > scratchWorkingDirectory=3Dfoo on host A and scratchWorkingDirectory=3Dbar= on > hostB, all new sub-directories will be created. What is the precise > difference from this behaviour? > > Suresh > > [1] - > https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-sche= ma/src/main/resources/schemas/ApplicationDeploymentDescription.xsd > > On Jan 13, 2014, at 9:53 AM, Lahiru Gunathilake wrote= : > > > Hi Saminda, > > > > You can do this easily by writing another Handler and make sure you cal= l > this handler after AppDescriptorCheckHandler. You can have properties in > your handler configuration and read those properties and configure > different hosts with different base directory locations. So your handler > configuration could looks like this. > > > > > > AppDescriptorCheckHandler"/> > > class=3D"org.apache.airavata.gfac.handler.BaseDirectorySetupHandler"> > > value=3D"/home/trestles/base-directory"/> > > > > > > > > With handler properties you do not have to hard-code the host specific > directories or create another configuration file. > > > > Hope this helps ! > > > > Regards > > Lahiru > > > > > > On Fri, Jan 10, 2014 at 3:35 PM, Saminda Wijeratne > wrote: > > A specific gateway will have access to the directory location "foo" in = a > certain remote host A. Therefore the gateway would like to have all the > working directories for the jobs running in remote host A to be located > inside "foo". Similarly for jobs running in remote host B, all the workin= g > directories should be created inside the directory location "bar" of whic= h > the gateway has access. > > > > The reason why the working directories needs to be located in locations > accessible to the gateways is because the gateway may perform manual data > transfer. > > > > Thanks, > > Saminda > > > > On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh < > raminderjsingh@gmail.com> wrote: > > Hi Saminda, > > > > Do you want all the jobs should use same working directory? > AppDescriptorCheckHandler in GFAC handles working directory, input-output > directory in GFAC. You can control these configuration by customizing thi= s > input handler. Please describe the use case and i will be able to provide > better input. > > > > Thanks > > Raminder > > > > On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne > wrote: > > > >> Hi Devs, > >> > >> I want the parent directory of each working directory of the jobs to b= e > a predefined path defined for each remote host. What is the easiest way t= o > do this? > >> > >> Thanks, > >> Saminda > > > > > > > > > > > > -- > > System Analyst Programmer > > PTI Lab > > Indiana University > > --001a11c3bc4a7ede9604efddb86b Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi All,

Thanks for all the responses.

= Lahiru, I will look at implementing the class BaseDirectorySetupHandler.= Is there a way which the gateway id and user can be obtained inside th= e handler?

Suresh, this was also my initial thought. to use the applicationD= eploymentDescriptor properties. While it is a convenient place for administ= rators to specify since it's documented in schema (so to speak), they h= ave to do it for each application deployment and if the location changes th= ey have to manually update all of them. But since I'm aiming for a demo= at the moment this indeed is a good way to go as a short term solution.


On Mon,= Jan 13, 2014 at 11:14 AM, Suresh Marru <smarru@apache.org> = wrote:
Saminda, Lahiru,

Why not use the staticWorkingDirectory option within Application Deployment= Description [1]? The intention of this option was, within the provider, if= this is provided, it ignores the scratchWorkingDirectory and always uses a= static working directory location.

Also, if I re-read Saminda=92s original request, is it not the normal GFac = behavior? I mean, within the specified directory scratchWorkingDirectory=3D= foo on host A and scratchWorkingDirectory=3Dbar on hostB, all new sub-direc= tories will be created. What is the precise difference from this behaviour?=

Suresh

[1] - https://svn.apache.org/repos/asf/airavata/trunk/m= odules/commons/gfac-schema/src/main/resources/schemas/ApplicationDeployment= Description.xsd

On Jan 13, 2014, at 9:53 AM, Lahiru Gunathilake <glahiru@gmail.com> wrote:

> Hi Saminda,
>
> You can do this easily by writing another Handler and make sure you ca= ll this handler after AppDescriptorCheckHandler. =A0You can have properties= in your handler configuration and read those properties and configure diff= erent hosts with different =A0base directory locations. So your handler con= figuration could looks like this.
>
> <InHandlers>
> =A0 =A0 =A0 =A0 =A0 =A0 <Handler class=3D"org.apache.airavata.= gfac.handler. AppDescriptorCheckHandler"/>
> =A0 =A0 =A0 =A0 =A0 =A0 <Handler class=3D"org.apache.airavata.= gfac.handler.BaseDirectorySetupHandler">
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 <property name=3D"tres= tles" value=3D"/home/trestles/base-directory"/>
> =A0 =A0 =A0 =A0 =A0 =A0 </Handler>
>
> =A0 =A0 =A0 =A0 </InHandlers>
> With handler properties you do not have to hard-code the host specific= directories or create another configuration file.
>
> Hope this helps !
>
> Regards
> Lahiru
>
>
> On Fri, Jan 10, 2014 at 3:35 PM, Saminda Wijeratne <samindaw@gmail.com> wrote:
> A specific gateway will have access to the directory location "fo= o" in a certain remote host A. Therefore the gateway would like to hav= e all the working directories for the jobs running in remote host A to be l= ocated inside "foo". Similarly for jobs running in remote host B,= all the working directories should be created inside the directory locatio= n "bar" of which the gateway has access.
>
> The reason why the working directories needs to be located in location= s accessible to the gateways is because the gateway may perform manual data= transfer.
>
> Thanks,
> Saminda
>
> On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh <raminderjsingh@gmail.com> wrote:
> Hi Saminda,
>
> Do you want all the jobs should use same working directory? AppDescrip= torCheckHandler in GFAC handles working directory, input-output directory i= n GFAC. You can control these configuration by customizing this input handl= er. Please describe the use case and i will be able to provide better input= .
>
> Thanks
> Raminder
>
> On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <samindaw@gmail.com> wrote:
>
>> Hi Devs,
>>
>> I want the parent directory of each working directory of the jobs = to be a predefined path defined for each remote host. What is the easiest w= ay to do this?
>>
>> Thanks,
>> Saminda
>
>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University


--001a11c3bc4a7ede9604efddb86b--