commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Waitzman <...@bbn.com>
Subject Re: [commons-net] listFiles failing to parse Apr 26 02:45 like dates
Date Thu, 27 Apr 2006 14:33:49 GMT
To follow up on my own message, I found a workaround and here is a detailed 
analysis:

The issue is why does FTPClient.listFiles fail on just 4/26 02:xx times?
listFiles uses SimpleDateFormatter.parse which uses the Java date classes. 
Parsing fails when lenient=false for an invalid time according to daylight 
savings time.  According to a time zone information file, from 1967 to 1973 
the last Sunday in April was a transition date.  The date parsing code 
assumes the year is the unix epoch, e.g. 1970, when no year is explicit in 
the date string being parsed.  So Apr 26 1970 is a daylight savings 
spring-forward date and 2:00am to 2:59am does NOT exist on that date.

The fix I made was to force the FTP timezone to be UTC using:
    FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
    conf.setServerTimeZoneId("UTC");
    ftpClient.configure(conf);
The UTC timezone has no daylight savings time so all the times are valid.
I'd rather keep my default timezone but turn off daylight savings time 
transitions, but I don't know how to do that.  (Any suggestions?)

I made a test program which illustrates the problem
Lenient=true, TZ=Eastern
   Trying date Apr 26 02:04
     Parsed into Sun Apr 26 03:04:00 EDT 1970
   Trying date Apr 26 07:04
     Parsed into Sun Apr 26 07:04:00 EDT 1970
   Trying date Jul 26 02:04
     Parsed into Sun Jul 26 02:04:00 EDT 1970
Lenient=true, TZ=UTC
   Trying date Apr 26 02:04
     Parsed into Sat Apr 25 21:04:00 EST 1970
   Trying date Apr 26 07:04
     Parsed into Sun Apr 26 03:04:00 EDT 1970
   Trying date Jul 26 02:04
     Parsed into Sat Jul 25 22:04:00 EDT 1970
Lenient=false, TZ=Eastern
   Trying date Apr 26 02:04
     Parse failed with java.lang.NullPointerException
   Trying date Apr 26 07:04
     Parsed into Sun Apr 26 07:04:00 EDT 1970
   Trying date Jul 26 02:04
     Parsed into Sun Jul 26 02:04:00 EDT 1970
Lenient=false, TZ=UTC
   Trying date Apr 26 02:04
     Parsed into Sat Apr 25 21:04:00 EST 1970
   Trying date Apr 26 07:04
     Parsed into Sun Apr 26 03:04:00 EDT 1970
   Trying date Jul 26 02:04
     Parsed into Sat Jul 25 22:04:00 EDT 1970

The real bug is probably that listFiles Unix parser can't force the java 
data parsing code to assume the year is the current year when the 
UnixFTPEntryParser has to use the format string DEFAULT_RECENT_DATE_FORMAT. 
  Edge cases are complex here (ex. on January 2nd parsing December 31).

========== the test code =========
import java.text.*;
import java.util.Date;
import java.util.TimeZone;

public class date {

     static final String format1 = "MMM d HH:mm"; //Nov 9 20:06
     static final SimpleDateFormat formatter = new SimpleDateFormat(format1);

     public static void test(String str) {
         try {
             System.out.println("  Trying date " + str);
             ParsePosition pp = new ParsePosition(0);
             Date parsed = formatter.parse(str, pp);
             System.out.println("    Parsed into " + parsed.toString());
         } catch (Exception e) {
             System.out.println("    Parse failed with " + e);
         }
     }

     public static void testMultiple() {
         test("Apr 26 02:04");
         test("Apr 26 07:04");
         test("Jul 26 02:04");
     }

     public static void main(String [] args) {
         formatter.setLenient(true);

         TimeZone tz = TimeZone.getTimeZone("US/Eastern");
         formatter.setTimeZone(tz);

         System.out.println("Lenient=true, TZ=Eastern");
         testMultiple();

         tz = TimeZone.getTimeZone("UTC");
         formatter.setTimeZone(tz);

         System.out.println("Lenient=true, TZ=UTC");
         testMultiple();

         formatter.setLenient(false);

         tz = TimeZone.getTimeZone("US/Eastern");
         formatter.setTimeZone(tz);

         System.out.println("Lenient=false, TZ=Eastern");
         testMultiple();

         tz = TimeZone.getTimeZone("UTC");
         formatter.setTimeZone(tz);

         System.out.println("Lenient=false, TZ=UTC");
         testMultiple();
     }
};


========== =========

I previously wrote:
> We have used the FTPClient library for years.  Today it has started to 
> fail to parse a specific time of day pattern even though nothing has 
> changed in our code.  This happens across multiple ftp sites.
> 
> Example, in the following listings, the files marked on the right with 
> !! fail to parse (e.g. the array returned by listFiles has a null for them)
> ...
> -rw-r--r--    1 0        0            3208 Apr 26 09:30 gw.db.gz
> -rw-r--r--    1 0        0           60310 Apr 26 02:49 jpirr.db.gz !!
> -rw-r--r--    1 0        0            3972 Apr 25 16:37 koren.db.gz
...
> -rw-r--r--    1 0        0            1775 Apr 26 02:34 openface.db.gz !!
...
> -rw-r--r--    1 0        0           12581 Apr 26 02:16 sprint.db.gz !!


> -rw-r--r--    1 ftp      ftp      13968226 Apr 26 00:18 
> oix-full-snapshot-2006-04-26-0000.dat.bz2
> oix-full-snapshot-2006-04-26-0200.dat.bz2                !!
> -rw-r--r--    1 ftp      ftp            72 Apr 26 02:18 
> oix-full-snapshot-2006-04-26-0200.dat.md5                !!
> 
> It seems to be a sensitive to "Apr 26 02".  Very strange.
> I am using both commons-net 1.4.0 and 1.4.1 from multiple OSs with sun 
> 1.5.0_06.



-- 
  David Waitzman
  BBN Technologies

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message