oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Bennett <lmzxq....@gmail.com>
Subject Re: Problem happened when I tried to run the script "crawler_launcher"
Date Mon, 13 Aug 2012 11:58:06 GMT
Hey YunHee and Chris,

Sorry I've been out of the office for the last week and I've not gotten
around to check my emails yet :).

YunHee, apologies for the stale documentation relating to the crawler. I'll
head over there and update the guide to reflect the new interface when I
get some free time.

Thanks for trying in out - I'll post an update once I've fixed the
documentation.

Cheers,
Tom

On 8 August 2012 22:49, Mattmann, Chris A (388J) <
chris.a.mattmann@jpl.nasa.gov> wrote:

> Hi YunHee,
>
> Sorry, we need to update the docs, that is for sure. Can you help
> us remember by filing a JIRA issue to update the Crawler user
> guide and to fix the URL there?
>
> As for crawlerId, yes it's obsolete, you can find the modern
> 0.4 and 0.5-trunk options by running ./crawler_launcher -h
>
> Cheers,
> Chris
>
> On Aug 7, 2012, at 7:03 AM, YunHee Kang wrote:
>
> > Hi Chris and Sheryl,
> >
> > I understood  my mistake after modifying a wrong URL with the "/".
> > But there is the wrong  URL  that is used  as an option of
> > crawler_launcher in the apache oodt
> > homepage(http://oodt.apache.org/components/maven/crawler/user/).
> > --filemgrUrl http://localhost:9000/ \
> > So it made me confused.
> >
> > I tried to run the command mentioned below  according to  the home
> > page of apache oodt.
> > $ ./crawler_launcher --crawlerId MetExtractorProductCrawler
> > ERROR: Invalid option: 'crawlerId'
> >
> > But the error described above  was occurred.
> > Is the option 'crawlerid'  obsolete ?
> >
> > Thanks,
> > Yunhee
> >
> >
> > 2012/8/7 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
> >> Perfect, Sheryl, my thoughts exactly.
> >>
> >> Cheers,
> >> Chris
> >>
> >> On Aug 6, 2012, at 10:01 AM, Sheryl John wrote:
> >>
> >>> Hi Yunhee,
> >>>
> >>> Check out this OODT wiki for crawler :
> >>> https://cwiki.apache.org/confluence/display/OODT/OODT+Crawler+Help
> >>>
> >>> Did you try giving 'http://localhost:8000' without the "/" in the end?
> >>> Also, specify
> 'org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory'
> >>> for  'clientTransferer' option.
> >>>
> >>>
> >>> On Mon, Aug 6, 2012 at 9:46 AM, YunHee Kang <yunh.kang@gmail.com>
> wrote:
> >>>
> >>>> Hi Chris,
> >>>>
> >>>> I got an error message when I tried to run crawler_launcher by using
a
> >>>> shell script. The error message may be caused by a  wrong URL of
> >>>> filemgr.
> >>>> $ ./crawler_launcher.sh
> >>>> ERROR: Validation Failures: - Value 'http://localhost:8000/' is not
> >>>> allowed for option
> >>>> [longOption='filemgrUrl',shortOption='fm',description='File Manager
> >>>> URL'] - Allowed values = [http://.*:\d*]
> >>>>
> >>>> The following is the shell script that I wrote:
> >>>> $ cat crawler_launcher.sh
> >>>> #!/bin/sh
> >>>> export STAGE_AREA=/home/yhkang/oodt-0.5/cas-pushpull/staging/TESL2CO2
> >>>> ./crawler_launcher \
> >>>>      -op --launchStdCrawler \
> >>>>      --productPath $STAGE_AREA\
> >>>>      --filemgrUrl http://localhost:8000/\
> >>>>      --failureDir /tmp \
> >>>>      --actionIds DeleteDataFile MoveDataFileToFailureDir Unique \
> >>>>      --metFileExtension tmp \
> >>>>      --clientTransferer
> >>>> org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer
> >>>>
> >>>> I am wondering if there is a problem in the URL of the filemgr or
> elsewhere
> >>>>
> >>>> Thanks,
> >>>> Yunhee
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> -Sheryl
> >>
> >>
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> Chris Mattmann, Ph.D.
> >> Senior Computer Scientist
> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> Office: 171-266B, Mailstop: 171-246
> >> Email: chris.a.mattmann@nasa.gov
> >> WWW:   http://sunset.usc.edu/~mattmann/
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> Adjunct Assistant Professor, Computer Science Department
> >> University of Southern California, Los Angeles, CA 90089 USA
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message