oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Padams <jordan.pad...@gmail.com>
Subject Re: PushPull HTTP Protocol
Date Mon, 11 Nov 2013 02:25:35 GMT
Hey Lewis,

Thanks for the reply!

I had the following mime type specified before:

        <mime-type type="product/txt">
                <glob pattern="*.TXT"/>
        </mime-type>

I updated the config per your notes, but still got the following exception
(same as before):

*org.apache.oodt.cas.protocol.exceptions.ProtocolException: Failed to get
appropriate protocol for RemoteSite: alias = 'msl'  url =
'http://pds-imaging.jpl.nasa.gov <http://pds-imaging.jpl.nasa.gov>'
 username = 'none' cdTestDir = 'null' maxConnections = '-1'*


I didn't specify a runInfo element because, according to the docs, if I do
not specify a runInfo, then the daemon will run only once, and then quit.
 This is the desired functionality.

The log of the most current run is attached (cas-pushpull0.log).

On another note, I successfully ran the url-downloader script that ships
with PushPull (see updated version below, current used HttpClient versus
HttpProtocol) to download the file.  At least this shows the HttpProtocol
can get to the file.

./url-downloader
http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/CATALOG/CATINFO.TXT .

url-downloader

#!/bin/csh


$JAVA_HOME/bin/java -Djava.ext.dirs=../lib \
        -Djava.util.logging.config.file=../etc/logging.properties \
        org.apache.oodt.cas.protocol.http.HttpProtocol \
        --url $1 \
        --downloadToDir $2


If someone had an example of a configuration for HTTP that may help me
debug the issue further, but right now I'm not sure what the problem is.

Thanks,
Jordan



On Sat, Nov 9, 2013 at 2:08 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Hi Jordan,
> A couple of things here.
> * Firstly assuming we use "
> http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/CATALOG/" as the
> root path
> * AFAIK you would need to include policy/mimetypes.xml with a mapping
> something like the following
>
> <mime-info>
>     <mime-type type="metadata/cas_pushpull">
>         <glob pattern="*.info.tmp"/>
>     </mime-type>
>     <mime-type type="metadata/cas_metadata">
>         <glob pattern="*.cas"/>
>         <glob pattern="*.met"/>
>     </mime-type>
>     <mime-type type="product/pds-imaging">
>         <_comment>Description of the CATALOG directory contents for the
> MSL HAZCAM EDR PDS Archive</_comment>
>         <glob pattern="CATINFO.TXT" isregex="false"/>
>     </mime-type>
> </mime-info>
>
> * The ExternalSources.xml file you posted looks A OK.
> * RemoteSpecs.xml also looks AOK however I would also consider possibly
> adding the <runInfo> elements e.g. <runInfo
> firstRunDateTime="2013-11-09T00:00:00Z" period="1m" runOnReboot="yes"/> as
> this lets you specify a sleep/wait time for the daemon. 3 mins is quite
> long.
> * msl.xml looks fine from what I can see.
> Do you get any logging for the failed jobs?
> There is quite a bit of config to do here and I find that it is easy to
> make mistakes.
> Thanks
> Lewis
>
> On Sat, Nov 9, 2013 at 8:31 AM, Jordan Padams <jordan.padams@gmail.com>wrote:
>>
>>
>>
>> Thanks,
>> Jordan
>>
>>
>> --
>> Jordan Padams
>> Software Engineer
>> NASA Jet Propulsion Laboratory
>>
>
>
>
> --
> *Lewis*
>



-- 
Jordan Padams
Software Engineer
NASA Jet Propulsion Laboratory

Mime
View raw message