airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marlon Pierce <>
Subject Re: Partial results of an application run
Date Fri, 19 Sep 2014 16:18:37 GMT
Great feedback, Mark--


On 9/19/14, 12:03 PM, Miller, Mark wrote:
> Hi Raminder,
> If I understand the issue, I have a comment.
> The stdout/stderr files are absolutely critical to a gateway runner and the every end
user for debugging issues, even simple ones like a typo that breaks the infile formatting.
> There are two levels of benefit: first, the savvy user reads these docs and solves their
own problem more quickly, if it is on their side, and notifies me of the message when it isn't
on their side. This saves time for everyone.
> Second, when the non-savvy user reports an issue but can't figure out what's wrong,,
this is the first place I look to identify the issue. It also makes it possible at all to
debug on a reasonably large user population. removing these files takes away the levers both
users and gateway owners have to manage issues. If I understand the issue correctly, I don't
think that SciGap wants to inherit the job of debugging job runs for all its constituent Gateways,
and not passing these files along would do just that.
> I strongly feel, again assuming I understand correctly, that all the files available
from every failed job should be passed along to the Gateway by SciGap. If the Gateway owners
wish to debug every user issue on their own, they can pass only certain files along to the
> In our time with CIPRES I think I have used almost every file we or the users job has
created to debug an issue at one time or another.
> Sometimes the absence of a file alone tells me what the issue is. On the other hand,
many of code produce both STDOUT as a file, and stdout.txt as a file.
> If SciGap wanted to be responsible for eliminating that ambiguity, that might be fine.
But delivering both copies puts the effort back on the Gateway developer to decide how to
handle it, and perhaps that is again
> the best solution. And it requires no effort on the part of SciGap, which already has
many things to take care of.
> So my vote is to return everything to the Gateway for every job.
> The exception with this might be SciGap-created files that do not have any relevance
to the Gateway. I still think they would be of benefit to the Gateway developer (at least)
because then they can report the issue to SciGap directly and explicitly.
> That means the admin on the SciGap side does not have to look up the job, maneuver to
the directory, and open files to find the error message. It conserves many keystrokes/clicks.
If we have 50 client Gateways, we will be grateful when a User reports the SciGap error message
and job number, rather than just saying they have an issue.
> Those are my thoughts.
> Best,
> Mark
> From: Raminder Singh []
> Sent: Friday, September 19, 2014 7:56 AM
> To:
> Subject: Partial results of an application run
> Hi Dev,
> Currently we are not moving partial results (stdout/stderr and other files) to the gateway
incase of application failure (failed to produce output). This can be fixed but question is
do we want it to be the default behavior or based on some user flag in experiment. To make
it work properly, we need user input to provide regular expressions or other details about
required files, incase of failure. Any suggestions on these changes in Application catalog
and Airavata API. We also need APIs functions to get job working directory and other details.
I created a JIRA for this.
> Thanks
> Raminder

View raw message