RE: [Myexperiment-discuss] "Intelligent" storage of workflows

[I . H . C . Wassink]

Dear All,

Last week, We (Me together with many other people, among others, Katy, Marco, 
Pieter) have submitted a paper to a workshop that is about analysing workflows 
stored at myExperiment. The funny part is that it is implemented as a workflow 
too!

The workflow download the xml of the workflows stored at myExperiment and 
analyses the tasks that are used (BioMoby, Biomart etc). The workflow further 
analyses the amount of tasks spend to local workers, web services, scripting 
and sub-workflows. An R script is used to perform the statistical analysis. The 
workflow makes use of the Xpath processor to easily collect the information 
about the processors.

It will be easy to adapt the workflow to collect the data-links between these 
processors. 

An advantage of implementing your analysis as a workflow too is that you can 
share it, but also refer to it in your papers (the referee and others are able 
to validate your analysis).

The workflow can be found at:
http://www.myexperiment.org/workflows/648

I hope this will be helpful.

Kind regards,

Ingo



-----Original Message-----
From: Marco Roos [mailto:address@hidden 
Sent: woensdag 18 februari 2009 17:02
To: Paolo Missier
Cc: Alan R Williams; address@hidden; Wassink, I. (Ingo); Edgar Meij; M. Scott 
Marshall
Subject: Re: [Myexperiment-discuss] "Intelligent" storage of workflows

Hi Alan and others,

My two cents:

David's RDF work is very important for all sorts of activities that try to link 
and provide useful information from scientific experiments. RDF I think allows 
a structured, yet flexible way to query information, useful for distributed 
queries, meta-analysis, and to build new applications on.

RDF is not a requirement per sé for all analyses. For your information:
1. my colleague Edgar Meij made a workflow to create an index from what he 
could obtain via the myExperiment REST API; this allows us to perform
(Lucene) queries to retrieve content from myExperiment. We used this in a 
(proof-of-principle) application. If the myExperiment API provides a look 
inside workflows, it may be possible to retrieve the desired information via a 
personalised index.
2. Ingo Wassink just submitted a paper where he analysed the contents of 
workflows in myExperiment, again via a workflow.

Both are on cc; I'm sure they are willing to provide more information upon 
request.

Cheers,
Marco.

Paolo Missier wrote:
> Hi,
> Marco should be also interested in Don's answer -- in general, 
> primitives for mining workflows off myExperiment are going to be 
> useful soon (i.e., in combination with dynamic provenance) -Paolo
>> Hi,
>>
>> The API allows you to query for the separate components of a workflow, e.g.:
>>
>>   http://www.myexperiment.org/workflow.xml?id=172&elements=components
>>
>> This is available for Taverna 1 workflows at present.  If Taverna 2 
>> beta workflows are important for this, then I can prioritise getting 
>> that available.
>>
>> David Newman will correct me if I'm wrong, but this breakdown hasn't 
>> made it into the RDF yet, so right now you would have to trawl 
>> myExperiment live via the API for all the workflow components.
>>
>> It's a query that looks like SPARQL might be the best way forward.  
>> The myExperiment SPARQL endpoint is at:
>>
>>   http://rdf.myexperiment.org/sparql
>>
>> So I think David Newman and myself are probably the best people to 
>> talk to about doing this.
>>
>> Regards,
>> Don.
>
>
> Alan R Williams wrote:
>> Hello,
>>
>> I'm at the Taverna workshop.  One of the Taverna users is interested 
>> in the storage of workflows so that they can be queried for their 
>> content e.g. what processors are used, what connections are made.  
>> For
>> example:
>>
>>     display all workflows that have a connection between a blast
>>     and an alignment service
>>
>> Does anyone here know of where to look for ideas on how to do this, 
>> or people for him to co-operate with?
>>
>> His e-mail is address@hidden - but I'm interested in the 
>> answers as well :-)
>>
>> Thanks in advance,
>>
>> Alan
>>   
>
>
>
> _______________________________________________
> Myexperiment-discuss mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/myexperiment-discuss

-- 

Marco Roos
Faculty of Science
University of Amsterdam
Kruislaan 403, room F1.02
1098 SJ Amsterdam
tel. +31 (0) 20 525 7522
http://home.medewerker.uva.nl/m.roos1 (includes links to social networks) Note 
the change of e-mail address to address@hidden