commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brent Worden <brent.wor...@gmail.com>
Subject Re: [math] JDBC output to generate statistical results.
Date Tue, 26 Aug 2014 15:00:04 GMT
Another alternative is to use a
org.apache.commons.math3.stat.descriptive.DescriptiveStatistics
object to collect all the data and then use it to compute the summary
statistics you need.  Using it alleviates the need for doing all explicit
type casting and conversion:

DescriptiveStatistics ds = new DescriptiveStatistics();
while(rset.next()) {
    int observation = rset.getInt("M_SecondsAtStatus");
    ds.addValue(observation);
}

System.out.println("min: " + ds.getMin());
System.out.println("max: " + ds.getMax());
...

HTH,

Brent


On Tue, Aug 26, 2014 at 9:41 AM, Bernd Eckenfels <ecki@zusammenkunft.net>
wrote:

> Hello,
>
> First of all: Your DBMS might have SQL methods to calculate typical
> aggregates. This is not only easier to program, but also most likely faster
> and less resource intensive than doing it in an extra application.
>
> But since this is the commons list: If You want to use the Commons Math
> functions you have to present the set of values (in your case as an array).
> And since there is no adapter for result sets (I think) building the array
> would be done inside the loop. The most natural thing is to use an
> ArrayList to append the values in the loop,  but then you have to convert
> the resulting Double[] into double[]. The ArrayUtils in Apache Commons Lang
> could do that (but if you need to process millions of numbers it is not the
> most efficient way to do it).
>
> untested:
>
> ArrayList<Double> times = new ArrayList<>();
> while(rset.next()) {
>   times.add(Double.valueOf(rset.getDouble(T));
> }
> double timesArray[] = ArrayUtils.toPrimitive(times.toArray());
>
> And then you can use this array for the Math statistics.
>
> Gruss
> bernd
>
>
> --
> http://bernd.eckenfels.net
>
> ----- Ursprüngliche Nachricht -----
> Von: "Kulpanowski, David" <DKulpanowski@leegov.com>
> Gesendet: ‎26.‎08.‎2014 15:55
> An: "Commons Users List" <user@commons.apache.org>
> Betreff: RE: [math] JDBC output to generate statistical results.
>
> Thank you Mr. Ritter:
>
> Two issues:
> 1.) I am attempting to obtain univariate statistics from thousands of
> ambulance responses. For example, ambulance responses (in seconds) 534,
> 678, 943, 194 would be a mean of 587 seconds. Not by row, but rather as
> summary statistics.
> 2.) It appears that Apache Commons Math is needing a Double value. So I
> change it as shown below.
> Note on 2) Even though I am needing summary statistics I move the lines of
> code into the loop just to see what would happen.I just want to get it to
> work because it appears the problem is the type of variable (int, double,
> array).
>
> while (rset.next())
>                         {
>                                 double values =
> rset.getDouble("M_SecondsAtStatus");
>                                 System.out.println(values);
>                                 System.out.println("min: " +
> StatUtils.min(values));
>                                 System.out.println("max: " +
> StatUtils.max(values));
>                                 System.out.println("mean: " +
> StatUtils.mean(values));
>                                 System.out.println("product: " +
> StatUtils.product(values));
>                                 System.out.println("sum: " +
> StatUtils.sum(values));
>                                 System.out.println("variance: " +
> StatUtils.variance(values));
>                         }
>
> A red underline in Eclipse shows up and my mouse hovers over it. The error
> message is the following:
>
> "The method min(double[]) in the type StatUtils is not applicable for the
> arguments (double)"
>
> I then change the values variable to double[] as shown below:
>
> "double[] values = rset.getDouble("M_SecondsAtStatus");"
>
> java doesn't like this either. It gives a red underlined error message:
> "Type mismatch: cannot convert from double to double[]"
>
>
> I guess this boils down to two questions:
> 1.) How do I output a double[] array from database output?
> 2.) How do I output this double[] into a variable that Apache Commons Math
> will accept?
> ok, maybe three questions:
> 3.) Other people are using Apache Commons Math to understand their
> database data better. How are they doing it? A lot of guys have massive
> mainframe databases filled with health care data etc. They are doing
> sophisticated math with their data. How are they doing it?
>
> -----Original Message-----
> From: Benedikt Ritter [mailto:britter@apache.org]
> Sent: Tuesday, August 26, 2014 9:15 AM
> To: Commons Users List
> Subject: Re: [math] JDBC output to generate statistical results.
>
> > In you're code the variable values is defined within the scope of the
> while loop.
>
> D'oh worst of typos... should be "in your code" of corse ;-)
>
> 2014-08-26 15:13 GMT+02:00 Benedikt Ritter <britter@apache.org>:
>
> > Hello David,
> >
> > the problem you're encountering is a problem with scopes. A variable
> > is only available in the scope it was defined. In you're code the
> > variable values is defined within the scope of the while loop. This
> > means, that the variable is only defined between the curly brackets of
> the while loop.
> >
> > Your System.out statements try to access the values variable, which is
> > no longer accessible, since the flow of control has already left the
> > scope it was definied in (by finishing the iteration over the ResultSet).
> >
> > What you need to do is move the other System.out statements into the
> > loop like so:
> >
> >
> >                     while (rset.next())
> >                     {
> >                            int values = rset.getInt("M_SecondsAtStatus");
> >                            System.out.println(values);
> >
> >
> >                           // I am hoping to derive useful statistics
> > from my database, such as
> >                          // the following.this uses Jakarta Commons Math
> >                         System.out.println("min: " +
> > StatUtils.min(values));
> >                         System.out.println("max: " +
> > StatUtils.max(values));
> >                         System.out.println("mean: " +
> > StatUtils.mean(values));
> >                         System.out.println("product: " +
> > StatUtils.product(values));
> >                         System.out.println("sum: " +
> > StatUtils.sum(values));
> >                         System.out.println("variance: " +
> > StatUtils.variance(values));
> >                     }
> >
> >
> > This way statistics will be printed for each row in the result set.
> >
> > Regards,
> > Benedikt
> >
> > P.S.: Jakarta is an old name, that is not used any more. The name of
> > the project now is simple Apache Commons and you're using Apache Commons
> Math.
> >
> >
> > 2014-08-26 15:03 GMT+02:00 Kulpanowski, David <DKulpanowski@leegov.com>:
> >
> > Using jdbc I am querying my database of ambulance response times. My
> > goal
> >> is to take the output and process it into statistics using Jakarta
> >> Commons Math library. So far I am successful in querying my database
> >> and outputting the response times to the console. My next step is to
> >> process this output statistically, such as mean, medians, mode, etc.
> This is where I am stuck.
> >> What I can't figure out is how to get my database output into a
> >> format for Commons Math to generate a statistical analysis. In other
> >> words, I have
> >> 100,000 ambulance responses, now I want to do more advanced
> >> statistical analysis with this data.
> >> Shown below is my code.
> >>
> >> package javaDatabase;
> >>
> >> import java.sql.*;
> >> import org.apache.commons.math3.stat.StatUtils;
> >>
> >> public class javaConnect4
> >> {
> >>        public static void main(String[] args)
> >>        {
> >>              Connection conn = null;
> >>              Statement stmt = null;
> >>              try
> >>              {
> >>                     conn = DriverManager
> >>
> >>
> .getConnection("jdbc:sqlserver://myServerAddress;database=myDatabase;integratedsecurity=false;user=myUser;password=myPassword");
> >>                     stmt = conn.createStatement();
> >>                     String strSelect = "SELECT M_SecondsAtStatus FROM
> >> MManpower WHERE M_tTime > 'august 25, 2014' AND M_Code = 'USAR'";
> >>
> >>                     ResultSet rset = stmt.executeQuery(strSelect);
> >>
> >>                     while (rset.next())
> >>                     {
> >>                            int values =
> rset.getInt("M_SecondsAtStatus");
> >>                            System.out.println(values);
> >>                     }
> >>
> >>                     // I am hoping to derive useful statistics from
> >> my database, such as
> >>                     // the following.this uses Jakarta Commons Math
> >>                     System.out.println("min: " + StatUtils.min(values));
> >>                     System.out.println("max: " + StatUtils.max(values));
> >>                     System.out.println("mean: " +
> StatUtils.mean(values));
> >>                     System.out.println("product: " +
> >> StatUtils.product(values));
> >>                     System.out.println("sum: " + StatUtils.sum(values));
> >>                     System.out.println("variance: " +
> >> StatUtils.variance(values));
> >>
> >>              } catch (SQLException ex)
> >>              {
> >>                     ex.printStackTrace();
> >>              } finally
> >>              {
> >>                     try
> >>                     {
> >>                            if (stmt != null)
> >>                                  stmt.close();
> >>                            if (conn != null)
> >>                                  conn.close();
> >>                     } catch (SQLException ex)
> >>                     {
> >>                            ex.printStackTrace();
> >>                     }
> >>              }
> >>        }
> >> }
> >>
> >>
> >> An error message pops up in Eclipse and the variable "values" is red
> >> underlined; "values cannot be resolved to a variable".
> >> I am not sure how to get this to work.
> >> I don't understand how to output my ambulance response times from the
> >> database into something Apache Commons math will understand.
> >> How can I get Apache Commons math to take the output from my database
> >> and generate a statistical result?.
> >>
> >>
> >> NOTES:
> >> 1.) I have cross-posted this question on StackOverflow.com but have
> >> not resolved the issue.
> >> 2.) I have verified that Apache Commons Math is registered in my
> >> project by hand coding a small array and using Commons Math to generate
> statistics.
> >> So Apache Math works and my database output goes to the console
> >> window, so it works also. But how do you get them to work together?
> >> 3.) I am a geographer, not a computer programmer. Believe me, you
> >> cannot make it simple enough. Please be explicit in your answers.
> >>
> >> David Kulpanowski
> >> Database Analyst
> >> Lee County EMS
> >> PO Box 398
> >> Fort Myers, FL 33902-0398
> >> 239-533-3962
> >> DKulpanowski@Leegov.com
> >> Longitude: -81.861486
> >> Latitude: 26.528843
> >>
> >>
> >> ________________________________
> >> Please note: Florida has a very broad public records law. Most
> >> written communications to or from County Employees and officials
> >> regarding County business are public records available to the public
> and media upon request.
> >> Your email communication may be subject to public disclosure.
> >>
> >> Under Florida law, email addresses are public records. If you do not
> >> want your email address released in response to a public records
> >> request, do not send electronic mail to this entity. Instead, contact
> >> this office by phone or in writing.
> >>
> >
> >
> >
> > --
> > http://people.apache.org/~britter/
> > http://www.systemoutprintln.de/
> > http://twitter.com/BenediktRitter
> > http://github.com/britter
> >
>
>
>
> --
> http://people.apache.org/~britter/
> http://www.systemoutprintln.de/
> http://twitter.com/BenediktRitter
> http://github.com/britter
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message