commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alastair Young" <alastairyo...@metatv.com>
Subject RE: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat FTP server
Date Thu, 20 May 2004 16:40:49 GMT
I was in error. The value displayed is BLOCKS not BYTES. So adjusting
the REGEX to handle the "no slash" form should be the only adjustment
required.

Here is the response from the MadGoat FTP author:

>> What units does the HGFTP (2.5-3) display for file size when 
>> responding to a LIST? Is it using DIRECTORY/UNITS= or just DIRECTORY 
>> and having that pick up whatever the local SET PROCESS/UNITS setting 
>> is?
>
> The directory display is created by HGFTP itself, mimicking the output
of > DIRECTORY.  The file size shown is always the number of blocks
used.
 
>> We are having problems using the Java Jakarta Commons Net FTP client 
>> parsing listings from ftp.accuweather.com
<ftp://ftp.accuweather.com/> 
>> . The size is apparently being displayed as BYTES on this system and 
>> they are only doing a REGEX for bytes "used/allocated". 

> That was a while ago! 8-)  AFAIK, *all* of the VMS FTP servers display
the > file size only in blocks used.

> HGFTP can be configured to emulate a UNIX listing display, but you'd
have > to talk to the AccuWeather people about setting it up that way.

> Hunter
> ------
> Hunter Goatley, Process Software, http://www.process.com/
> <goathunter@GOATLEY.COM>     http://www.goatley.com/hunter/

Alastair Young
Senior Network Engineer
Technical Support & Operations
 M E T A T V
alastairyoung@metatv.com 
Tel : 415.380.6220 
Cell: 925.784.0812 

-----Original Message-----
From: Steve Cohen [mailto:scohen@javactivity.org] 
Sent: Wednesday, May 19, 2004 4:47 PM
To: Alastair Young
Cc: commons-dev@jakarta.apache.org
Subject: Re: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat
FTP server

Moving this discussion onto the commons-dev list, where it really
belongs.  
Mr. Young, if you're not a member, we'll keep you in it.

If I read you correctly, what we are doing is definitely wrong, but it
isn't 
clear what the right way should be.

IF there is a slash, an APPROXIMATE size in bytes is available by
multiplying 
the "numerator" of this "fraction" by 512.  But if there is no slash and
just 
a single number, it can be either blocks or bytes.  What is the right
thing 
to do in THAT case? ???

Not a pretty picture!

On Wednesday 19 May 2004 12:36 pm, you wrote:

> The MadGoat documentation indicates that it can serve listings in Unix
> form, but this is set up by a LOGICAL (aka environmental variable) on
> the server. This isn't our server, so we don't have the option of
making
> that change, and if we did, it would break our legacy production code
> which is pulling stuff off the same server.
>
>
>
> I haven't used VMS since 1990 (VMS 4.7?) but I have found the current
> VMS docs online at HP. See extracts below.
>
>
>
> Digging in the VMS documentation indicates that the size column
defaults
> to blocks "used/allocated" but other valid forms are "used",
"allocated"
> or "bytes". I have dropped a line to Hunter Goatley, author of MadGoat
> FTP, asking him if the server is specifying the format or just calling
> the DIRECTORY command and inheriting the setting. If the latter, we
can
> assume that no slash = BYTES, as the ALLOCATION and USED values would
> not occur.
>
>
>
> From the VMS documentation for the DIRECTORY command at
>
http://h71000.www7.hp.com/DOC/732FINAL/9996/9996pro_016.html#index_x_316
>
>
>
> /SIZE[=option]
>
> /NOSIZE (default)
>
> Displays the size in blocks of each file. If you omit the option
> parameter, the default lists the file size in blocks used (USED).
> Specify one of the following options:
>
>
> ALL
>
> Lists the file size both in blocks allocated and blocks used.
>
>
> ALLOCATION
>
> Lists the file size in blocks allocated.
>
>
> UNITS[=option]
>
> Allows you to override the current default specified by SET
> PROCESS/UNITS so that you can display file size in your choice of
blocks
> or bytes.
>
> The following keywords are valid options with the UNITS keyword:
BLOCKS,
> BYTES.
>
> If you specify UNITS with no option, the default value is not changed.
>
>
> USED
>
> Lists the file size in blocks used.
>
>
>
> Also for SET PROCESS
>
http://h71000.www7.hp.com/DOC/732FINAL/9996/9996pro_061.html#index_x_100
> 1
>
>
>
> /UNITS[=keyword]
>
> Specifies whether the amount of disk space reported by certain
utilities
> is to be displayed in blocks or bytes. Keyword options are:
>
>
> Option
>
> Description
>
>
> BLOCKS
>
> Displays disk space in blocks.
>
>
> BYTES
>
> Displays disk space in bytes.
>
> Blocks is the default until /UNITS is set to BYTES. If you specify
> /UNITS with no keyword, disk space is reported in blocks.
>
> Displays that are affected by changing the value of /UNITS include
> output from certain forms of the following commands: COPY, DELETE,
> DIRECTORY, PURGE, SHOW DEVICE, SHOW MEMORY, and SHOW QUOTA. Note that
> input to these commands can be specified only in blocks. The
DIRECTORY,
> SHOW DEVICES, and SHOW MEMORY commands have a qualifier that lets you
> override the default SET PROCESS/UNITS setting for a single command.
>
>
>
> Alastair Young
>
> Senior Network Engineer
>
> Technical Support & Operations
>
>  M E T A T V
>
> alastairyoung@metatv.com
>
> Tel : 415.380.6220
>
> Cell: 925.784.0812
>
>
>
> -----Original Message-----
> From: Ojeda, Winston [mailto:Winston.Ojeda@qg.com]
> Sent: Wednesday, May 19, 2004 6:11 AM
> To: Steve Cohen; Alastair Young; scohen@apache.org; sestegra@free.fr
> Subject: RE: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat
> FTP server
>
>
>
> When I wrote the VMSListParser many years ago it was more in terms of
a
>
> test to see what the Netcomponents library could do for us.
>
> Little did I know, wound up being the pillar of everything we do
around
>
> here.
>
>
>
> Anyways, you are right. Under the VMS Operating System, every file is
>
> described in terms of blocks for size. One block is exactly 512 bytes.
>
> A regular directory listing will show two values; ie. 2709/2715.
>
> The nominator, which is what the parser uses, is the used blocks out
of
>
> the allocated blocks which is the denominator.
>
> Now, as you can see, the problem with the current implementation of
the
>
> ListParser for VMS is that to find "bytes" you will need to multiply
the
>
> numerator by 512 which will net you a upward rounded number. This is
an
>
> aproximation and cannot/should not be trusted for critical operations.
>
>
>
> The only way to do this right, that I know of, is to find the last
used
>
> block and see how many bytes are populated in it. That will net you
the
>
> "true" size of the file. The only way we have found is through a very
>
> complex data server written in java to provide access to native VMS
>
> calls.
>
>
>
> Maybe someone else can come up with a better way to do this.
>
>
>
> I don't know about MadGoat too much, but some other VMS ftp servers
>
> already serve directory listings in UNIX mode. If MadGoat is indeed
>
> serving a size in bytes, the VMSListParser can be modifed to accept
both
>
> formats.
>
>
>
>
>
>
>
> Winston Ojeda
>
> Software Engineer
>
> Information Systems
>
> West Allis, Wisconsin
>
> winston.ojeda@qg.com
>
> www.QG.com
>
>
>
>
>
> -----Original Message-----
>
> From: Steve Cohen [mailto:scohen@javactivity.org]
>
> Sent: Tuesday, May 18, 2004 11:09 PM
>
> To: Alastair Young; Ojeda, Winston; scohen@apache.org;
sestegra@free.fr
>
> Subject: Re: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat
>
> FTP server
>
>
>
>
>
> I would say that yes, we can probably modify our regex to work the way
>
> this
>
> server does.  Basically we are just ignoring the second number.
Please
>
> log
>
> this as a bug in bugzilla.
>
>
>
> However, after glancing at this, I am a little concerned that we are
not
>
>
>
> correctly interpreting the size for the regular servers.  It seems
like
>
> the
>
> second number must mean something.  If the first number is indeed the
>
> size,
>
> then all of our original samples are pointing at very small files.  If
I
>
> had
>
> to bet, I would say that we're misinterpreting what we're parsing.
>
>
>
> Can you, Mr. Young, or someone else confirm the meaning of the two
>
> numbers in
>
> the regular VMS FTP listing?
>
> On Tuesday 18 May 2004 5:51 pm, Alastair Young wrote:
> > We are trying to port our current script based download from
> >
> > ftp.accuweather.com <ftp://ftp.accuweather.com/>  to a java app
using
> >
> > the commons net ftp client.
> >
> >
> >
> >
> >
> >
> >
> > Unfortunately the VMSFTPEntryParser does not match the file listings
> >
> > returned by the ftp server at that site.
> >
> >
> >
> >
> >
> >
> >
> > They are running MadGoat V2.5-3
> >
> >
> >
> >
> >
> >
> >
> > Connected to ftp.accuweather.com (207.242.93.48).
> >
> >
> >
> > 220 ftp.AccuWeather.com MadGoat FTP server V2.5-3 ready.
> >
> >
> >
> > Name (ftp.accuweather.com:ayoung): metatv
> >
> >
> >
> > 331 Username "metatv" Okay, need password.
> >
> >
> >
> > Password:
> >
> >
> >
> > 230-Welcome to AccuWeather, Inc.
> >
> >
> >
> > 230-User "METATV" logged in, 18-MAY-2004 22:04:16 +0000, proceed.
> >
> >
> >
> > 230 Connection closes if idle for 5 min.
> >
> >
> >
> > Remote system type is VMS.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > The file listings look like
> >
> >
> >
> >
> >
> >
> >
> > ftp> dir *.xml
> >
> >
> >
> > 227 Entering passive mode (207,242,93,48,17,149).
> >
> >
> >
> > 150 LIST of *.XML Started; Opening data connection.
> >
> >
> >
> >
> >
> >
> >
> > BBSDATA2:[METATV]
> >
> >
> >
> >
> >
> >
> >
> > 5DAY.XML;2615        21961 18-MAY-2004 21:17 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > HOROSCOPES.XML;146      16 18-MAY-2004 10:22 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > NEWENG_TIDES.XML;146
> >
> >
> >
> >                         17 18-MAY-2004 10:22 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > POLLEN.XML;704       12308 18-MAY-2004 10:23 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,)
> >
> >
> >
> > RISE.XML;146          1064 18-MAY-2004 10:22 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > SOAPS.XML;1             16 18-MAY-2004 10:22 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > USCUR_NEW_UVI.XML;14175
> >
> >
> >
> >                      17230 18-MAY-2004 21:12 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > US_CUR.XML;16136      7469 18-MAY-2004 21:13 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> > US_CUR_NEW.XML;16135
> >
> >
> >
> >                       7868 18-MAY-2004 21:13 [CLIENTS,METATV]
> >
> > (RWED,RWED,RE,RE)
> >
> >
> >
> >
> >
> >
> >
> > Total of 9 Files, 67949 Blocks.
> >
> >
> >
> > 226 File transfer Okay; Closing data connection.
> >
> >
> >
> > ftp>
> >
> >
> >
> >
> >
> >
> >
> > Looking at the columns it appears VMSFTPEntryParser is looking for
n/n
> >
> >
> >
> > in the second column and using the number before the slash as the
file
> >
> >
> >
> > size. The accuweather listing does not have the /n.
> >
> >
> >
> >
> >
> >
> >
> > I did VMS long ago - I *think* the n/n represents blocks
> >
> > used/allocated. The single number in the MadGoat is clearly in
bytes.
> >
> >
> >
> >
> >
> >
> >
> > Any way this FTP server can be supported?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Alastair Young
> >
> >
> >
> > Senior Network Engineer
> >
> >
> >
> > Technical Support & Operations
> >
> >
> >
> >  <http://www.metatv.com/>   <http://www.metatv.com/>
> >
> >
> >
> > alastairyoung@metatv.com <mailto:alastairyoung@metatv.com%20>
> >
> >
> >
> > Tel : 415.380.6220
> >
> >
> >
> > Cell: 925.784.0812


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message