airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emre Brookes <e...@biochem.uthscsa.edu>
Subject Re: Retrieving Experiment Summaries
Date Thu, 20 Mar 2014 23:56:50 GMT
You might want to have a look at mongodb's methods of find()ing 
distinct() and aggregate() data.
Similar, but different, layout and it provides some nice json such as { 
$in : { Experiment.Owner : ["bob","alice", ... ] }
and $gt : { Experiment.Runtime : 100 } etc so that logical operators 
could be incorporated.
Just wanted to point this out as a possible defined interface
.. it may or may not be worth the effort to provide some subset of these 
features.

-E.


Saminda Wijeratne wrote:
> In an offline discussion with Chathuri, we came up with a simple way 
> for gateway developers to specify retrieving a filtered set of 
> experiment data based on the requirements of the gateway user.
>
> eg:
>
> SearchQuery query =
> new SearchQuery({Experiment.Name, Experiment.Status}, 
> {{Experiment.Owner,"bob"},{Experiment.Project,"manhattan"}{Experiment.Created,"03-19-2014",">"})
> List<Experiment> experiments = thriftAPI.getExperiments(query);
>
> /Sample syntax/
> sq = new SearchQuery(<list of fields that needs to be filled>, <list 
> of filter criteria for the data>)
>
> Further more the SearchQuery will have the capability to specify 
> paging (eg; experiments from 11 to 20).
>
> wdyt?
>
> Saminda
>
>
>
> On Tue, Mar 18, 2014 at 3:04 PM, Lahiru Gunathilake <glahiru@gmail.com 
> <mailto:glahiru@gmail.com>> wrote:
>
>      On Tue, Mar 18, 2014 at 2:55 PM, Saminda Wijeratne
>     <samindaw@gmail.com <mailto:samindaw@gmail.com>> wrote:
>
>         For performance issues a gateway should only request a subset
>         of data of an experiment from Airavata server to compile a
>         summary view of the experiment to the scientist. Based on my
>         current experience I feel the following data is required to
>         compile a general summary.
>
>           - Exp ID/Name
>           - Status
>           - Project
>           - Owner/Creation time
>
>     +1, We can show minimum data and give detailed view on-demand. But
>     I think we need to support experiment search based on some
>     criteria and develop an index for each search criteria, because if
>     I ran jobs for 6 months and I would never want to get all my
>     experiments, even thought we make it super fast will minimum data.
>
>     ex: I want to search the experiments I ran last week, or with some
>     text base search.
>
>     We can use the above solution Saminda suggested in searching too.
>
>     Lahiru
>
>
>         We have seeing a direct relationship between the number of
>         experiment data records and the turnaround time. Thus we may
>         need some paging when requesting the experiment data.
>
>         wdyt? Your thoughts are welcome.
>
>         (Using JIRA [1] to track the status of this task)
>
>         A detailed discussion on the topic is on the Architecture
>         mailing list [2].
>
>         Regards,
>         Saminda
>
>         1. https://issues.apache.org/jira/browse/AIRAVATA-995
>         2.
>         http://www.mail-archive.com/architecture@airavata.apache.org/msg00080.html
>
>
>
>
>     -- 
>     System Analyst Programmer
>     PTI Lab
>     Indiana University
>
>


Mime
View raw message