commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernd Eckenfels <e...@zusammenkunft.net>
Subject Re: [math] JDBC output to generate statistical results.
Date Tue, 26 Aug 2014 16:36:57 GMT
Hello David,

if you download and include commons-lang3.jar in your classpath Eclipse
will recognize ArrayUtils and allow you to import
org.apache.commons.lang3.

Here is the Javadoc for it:
http://commons.apache.org/proper/commons-lang/javadocs/api-release/index.html

Greetings
Bernd

BTW: Commons Developers: I do wonder if this would be a good feature for
dbutils. It has currently a RowProcessor, but that works either in
Object[] or needs to map to beans. Returning a simple type array for a
single column might be usefull?


  Am Tue, 26 Aug 2014
11:37:12 -0400 schrieb "Kulpanowski, David" <DKulpanowski@leegov.com>:

> Messrs. Worden and Eckenfels:
> 
> Thank you both for your kind assistance. 
> 
> Mr. Worden: 
> your solution works perfectly. This is exactly what I am looking for. 
> 
> Mr. Eckenfels:
> Please excuse my lack of java coding skills. I am working on it by
> taking on projects at my job. I think your solution will work and I
> want to use it in my code because I am now going to use Apache
> Commons Math for more sophisticated statistics such as regression and
> hypothesis testing. For example, is the mean average ambulance
> response time in Cape Coral the statistically significantly different
> from the mean average response time in Fort Myers. I anticipate
> needing your code so I need to ask for additional help:
> 
> In the final line of code Eclipse is putting a red underline under
> ArrayUtils. 
> 
> ArrayList<Double> times = new ArrayList<>();
> while (rset.next())
> {
> times.add(Double.valueOf(rset.getDouble("M_SecondsAtStatus")));
> }
> double timesArray[] = ArrayUtils.toPrimitive(times.toArray());
> 
> My mouse hovers over it and the message is: "ArrayUtils cannot be
> resolved". Eclipse offers nine quick fixes:
> 1.) create class ArrayUtils.
> 2.) create constant ArrayUtils
> 3.) create local variable ArrayUtils
> 4.) change to ArgUtils
> 5.) change to Array
> 6.) change to Arrays
> 7.) create field ArrayUtils
> 8.) create parameter ArrayUtils
> 9.) fix project set up
> 
> Which one should I use to output my data in a format Apache Commons
> Math will utilize in its functions?
> 
> 
> -----Original Message-----
> From: Brent Worden [mailto:brent.worden@gmail.com] 
> Sent: Tuesday, August 26, 2014 11:00 AM
> To: Commons Users List
> Subject: Re: [math] JDBC output to generate statistical results.
> 
> Another alternative is to use a
> org.apache.commons.math3.stat.descriptive.DescriptiveStatistics
> object to collect all the data and then use it to compute the summary
> statistics you need.  Using it alleviates the need for doing all
> explicit type casting and conversion:
> 
> DescriptiveStatistics ds = new DescriptiveStatistics();
> while(rset.next()) {
>     int observation = rset.getInt("M_SecondsAtStatus");
>     ds.addValue(observation);
> }
> 
> System.out.println("min: " + ds.getMin());
> System.out.println("max: " + ds.getMax()); ...
> 
> HTH,
> 
> Brent
> 
> 
> On Tue, Aug 26, 2014 at 9:41 AM, Bernd Eckenfels
> <ecki@zusammenkunft.net> wrote:
> 
> > Hello,
> >
> > First of all: Your DBMS might have SQL methods to calculate typical 
> > aggregates. This is not only easier to program, but also most
> > likely faster and less resource intensive than doing it in an extra
> > application.
> >
> > But since this is the commons list: If You want to use the Commons 
> > Math functions you have to present the set of values (in your case
> > as an array). And since there is no adapter for result sets (I
> > think) building the array would be done inside the loop. The most
> > natural thing is to use an ArrayList to append the values in the
> > loop,  but then you have to convert the resulting Double[] into
> > double[]. The ArrayUtils in Apache Commons Lang could do that (but
> > if you need to process millions of numbers it is not the most
> > efficient way to do it).
> >
> > untested:
> >
> > ArrayList<Double> times = new ArrayList<>();
> > while(rset.next()) {
> >   times.add(Double.valueOf(rset.getDouble(T));
> > }
> > double timesArray[] = ArrayUtils.toPrimitive(times.toArray());
> >
> > And then you can use this array for the Math statistics.
> >
> > Gruss
> > bernd
> >
> >
> > --
> > http://bernd.eckenfels.net
> >
> > ----- Ursprüngliche Nachricht -----
> > Von: "Kulpanowski, David" <DKulpanowski@leegov.com>
> > Gesendet: ‎26.‎08.‎2014 15:55
> > An: "Commons Users List" <user@commons.apache.org>
> > Betreff: RE: [math] JDBC output to generate statistical results.
> >
> > Thank you Mr. Ritter:
> >
> > Two issues:
> > 1.) I am attempting to obtain univariate statistics from thousands
> > of ambulance responses. For example, ambulance responses (in
> > seconds) 534, 678, 943, 194 would be a mean of 587 seconds. Not by
> > row, but rather as summary statistics.
> > 2.) It appears that Apache Commons Math is needing a Double value.
> > So I change it as shown below.
> > Note on 2) Even though I am needing summary statistics I move the 
> > lines of code into the loop just to see what would happen.I just
> > want to get it to work because it appears the problem is the type
> > of variable (int, double, array).
> >
> > while (rset.next())
> >                         {
> >                                 double values = 
> > rset.getDouble("M_SecondsAtStatus");
> >                                 System.out.println(values);
> >                                 System.out.println("min: " + 
> > StatUtils.min(values));
> >                                 System.out.println("max: " + 
> > StatUtils.max(values));
> >                                 System.out.println("mean: " + 
> > StatUtils.mean(values));
> >                                 System.out.println("product: " + 
> > StatUtils.product(values));
> >                                 System.out.println("sum: " + 
> > StatUtils.sum(values));
> >                                 System.out.println("variance: " + 
> > StatUtils.variance(values));
> >                         }
> >
> > A red underline in Eclipse shows up and my mouse hovers over it.
> > The error message is the following:
> >
> > "The method min(double[]) in the type StatUtils is not applicable
> > for the arguments (double)"
> >
> > I then change the values variable to double[] as shown below:
> >
> > "double[] values = rset.getDouble("M_SecondsAtStatus");"
> >
> > java doesn't like this either. It gives a red underlined error
> > message: "Type mismatch: cannot convert from double to double[]"
> >
> >
> > I guess this boils down to two questions:
> > 1.) How do I output a double[] array from database output?
> > 2.) How do I output this double[] into a variable that Apache
> > Commons Math will accept?
> > ok, maybe three questions:
> > 3.) Other people are using Apache Commons Math to understand their 
> > database data better. How are they doing it? A lot of guys have 
> > massive mainframe databases filled with health care data etc. They
> > are doing sophisticated math with their data. How are they doing it?
> >
> > -----Original Message-----
> > From: Benedikt Ritter [mailto:britter@apache.org]
> > Sent: Tuesday, August 26, 2014 9:15 AM
> > To: Commons Users List
> > Subject: Re: [math] JDBC output to generate statistical results.
> >
> > > In you're code the variable values is defined within the scope of 
> > > the
> > while loop.
> >
> > D'oh worst of typos... should be "in your code" of corse ;-)
> >
> > 2014-08-26 15:13 GMT+02:00 Benedikt Ritter <britter@apache.org>:
> >
> > > Hello David,
> > >
> > > the problem you're encountering is a problem with scopes. A
> > > variable is only available in the scope it was defined. In you're
> > > code the variable values is defined within the scope of the while
> > > loop. This means, that the variable is only defined between the
> > > curly brackets of
> > the while loop.
> > >
> > > Your System.out statements try to access the values variable,
> > > which is no longer accessible, since the flow of control has
> > > already left the scope it was definied in (by finishing the
> > > iteration over the ResultSet).
> > >
> > > What you need to do is move the other System.out statements into
> > > the loop like so:
> > >
> > >
> > >                     while (rset.next())
> > >                     {
> > >                            int values =
> > > rset.getInt("M_SecondsAtStatus"); System.out.println(values);
> > >
> > >
> > >                           // I am hoping to derive useful
> > > statistics from my database, such as
> > >                          // the following.this uses Jakarta
> > > Commons Math System.out.println("min: " + 
> > > StatUtils.min(values));
> > >                         System.out.println("max: " + 
> > > StatUtils.max(values));
> > >                         System.out.println("mean: " + 
> > > StatUtils.mean(values));
> > >                         System.out.println("product: " + 
> > > StatUtils.product(values));
> > >                         System.out.println("sum: " + 
> > > StatUtils.sum(values));
> > >                         System.out.println("variance: " + 
> > > StatUtils.variance(values));
> > >                     }
> > >
> > >
> > > This way statistics will be printed for each row in the result
> > > set.
> > >
> > > Regards,
> > > Benedikt
> > >
> > > P.S.: Jakarta is an old name, that is not used any more. The name
> > > of the project now is simple Apache Commons and you're using
> > > Apache Commons
> > Math.
> > >
> > >
> > > 2014-08-26 15:03 GMT+02:00 Kulpanowski, David
> > > <DKulpanowski@leegov.com>:
> > >
> > > Using jdbc I am querying my database of ambulance response times.
> > > My goal
> > >> is to take the output and process it into statistics using
> > >> Jakarta Commons Math library. So far I am successful in querying
> > >> my database and outputting the response times to the console. My
> > >> next step is to process this output statistically, such as mean,
> > >> medians, mode, etc.
> > This is where I am stuck.
> > >> What I can't figure out is how to get my database output into a 
> > >> format for Commons Math to generate a statistical analysis. In 
> > >> other words, I have
> > >> 100,000 ambulance responses, now I want to do more advanced 
> > >> statistical analysis with this data.
> > >> Shown below is my code.
> > >>
> > >> package javaDatabase;
> > >>
> > >> import java.sql.*;
> > >> import org.apache.commons.math3.stat.StatUtils;
> > >>
> > >> public class javaConnect4
> > >> {
> > >>        public static void main(String[] args)
> > >>        {
> > >>              Connection conn = null;
> > >>              Statement stmt = null;
> > >>              try
> > >>              {
> > >>                     conn = DriverManager
> > >>
> > >>
> > .getConnection("jdbc:sqlserver://myServerAddress;database=myDatabase;i
> > ntegratedsecurity=false;user=myUser;password=myPassword");
> > >>                     stmt = conn.createStatement();
> > >>                     String strSelect = "SELECT M_SecondsAtStatus 
> > >> FROM MManpower WHERE M_tTime > 'august 25, 2014' AND M_Code = 
> > >> 'USAR'";
> > >>
> > >>                     ResultSet rset =
> > >> stmt.executeQuery(strSelect);
> > >>
> > >>                     while (rset.next())
> > >>                     {
> > >>                            int values =
> > rset.getInt("M_SecondsAtStatus");
> > >>                            System.out.println(values);
> > >>                     }
> > >>
> > >>                     // I am hoping to derive useful statistics
> > >> from my database, such as
> > >>                     // the following.this uses Jakarta Commons
> > >> Math System.out.println("min: " + StatUtils.min(values));
> > >>                     System.out.println("max: " +
> > >> StatUtils.max(values)); System.out.println("mean: " +
> > StatUtils.mean(values));
> > >>                     System.out.println("product: " + 
> > >> StatUtils.product(values));
> > >>                     System.out.println("sum: " +
> > >> StatUtils.sum(values)); System.out.println("variance: " + 
> > >> StatUtils.variance(values));
> > >>
> > >>              } catch (SQLException ex)
> > >>              {
> > >>                     ex.printStackTrace();
> > >>              } finally
> > >>              {
> > >>                     try
> > >>                     {
> > >>                            if (stmt != null)
> > >>                                  stmt.close();
> > >>                            if (conn != null)
> > >>                                  conn.close();
> > >>                     } catch (SQLException ex)
> > >>                     {
> > >>                            ex.printStackTrace();
> > >>                     }
> > >>              }
> > >>        }
> > >> }
> > >>
> > >>
> > >> An error message pops up in Eclipse and the variable "values" is 
> > >> red underlined; "values cannot be resolved to a variable".
> > >> I am not sure how to get this to work.
> > >> I don't understand how to output my ambulance response times
> > >> from the database into something Apache Commons math will
> > >> understand. How can I get Apache Commons math to take the output
> > >> from my database and generate a statistical result?.
> > >>
> > >>
> > >> NOTES:
> > >> 1.) I have cross-posted this question on StackOverflow.com but
> > >> have not resolved the issue.
> > >> 2.) I have verified that Apache Commons Math is registered in my 
> > >> project by hand coding a small array and using Commons Math to 
> > >> generate
> > statistics.
> > >> So Apache Math works and my database output goes to the console 
> > >> window, so it works also. But how do you get them to work
> > >> together? 3.) I am a geographer, not a computer programmer.
> > >> Believe me, you cannot make it simple enough. Please be explicit
> > >> in your answers.
> > >>
> > >> David Kulpanowski
> > >> Database Analyst
> > >> Lee County EMS
> > >> PO Box 398
> > >> Fort Myers, FL 33902-0398
> > >> 239-533-3962
> > >> DKulpanowski@Leegov.com
> > >> Longitude: -81.861486
> > >> Latitude: 26.528843
> > >>
> > >>
> > >> ________________________________
> > >> Please note: Florida has a very broad public records law. Most 
> > >> written communications to or from County Employees and officials 
> > >> regarding County business are public records available to the 
> > >> public
> > and media upon request.
> > >> Your email communication may be subject to public disclosure.
> > >>
> > >> Under Florida law, email addresses are public records. If you do 
> > >> not want your email address released in response to a public 
> > >> records request, do not send electronic mail to this entity. 
> > >> Instead, contact this office by phone or in writing.
> > >>
> > >
> > >
> > >
> > > --
> > > http://people.apache.org/~britter/
> > > http://www.systemoutprintln.de/
> > > http://twitter.com/BenediktRitter
> > > http://github.com/britter
> > >
> >
> >
> >
> > --
> > http://people.apache.org/~britter/
> > http://www.systemoutprintln.de/
> > http://twitter.com/BenediktRitter
> > http://github.com/britter
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message