Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C26B919D4B for ; Sat, 26 Mar 2016 22:16:44 +0000 (UTC) Received: (qmail 2207 invoked by uid 500); 26 Mar 2016 22:16:44 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 2094 invoked by uid 500); 26 Mar 2016 22:16:44 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 2083 invoked by uid 99); 26 Mar 2016 22:16:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 26 Mar 2016 22:16:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id DEABEC0217 for ; Sat, 26 Mar 2016 22:16:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id gZ6QldYujawh for ; Sat, 26 Mar 2016 22:16:38 +0000 (UTC) Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 5D78A5F240 for ; Sat, 26 Mar 2016 22:16:38 +0000 (UTC) Received: by mail-wm0-f51.google.com with SMTP id l71so2973560wmg.1 for ; Sat, 26 Mar 2016 15:16:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:subject:message-id:date:to:mime-version; bh=TZzpvb5ZCuAn6ZtOc/srdb1ZNUqY5p0oKT31wAVnd+c=; b=LayGhU2NKPg+Z5gu3rb24gzQjvsNnZf9v9L+VXgP4p69VGZJ7FvtOom3Buymy6n4Pp vkWsiEjmQvtmvr+NLpO+l7ay/ERzNdWHCpBpHTilOB4TMD0KCGVwIKz1zaahVzRSEZC8 Z/hGN0DX9aoyJ7DneLil9eo6r48RyBqSBRiJ2zTfxZ69jchdieq7vXCPLkda6wR3mPYC EMGaz2/IcA2Ab8gA7K0QwQGyz0QCAzinFJnDcNqorDJ1UvG9GAeWYYGM5FuKIw9w12VE +DW/CYxmwfyWmZflr5MdlpkP4HxVTn2n9dNZDuREaalIhINwNzMple0wAkfuBqLJQNRt 9KQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:subject:message-id:date:to:mime-version; bh=TZzpvb5ZCuAn6ZtOc/srdb1ZNUqY5p0oKT31wAVnd+c=; b=fJXd+9bcZlHrgGO2and9aCmqp7Rit5Kdbw8TY2TnCDHfDhv1kx83KSHx8cMFQ4//eu urwijDvcidGHCuflfYY2/LQTMaIWGFFzQomn6z6VdjCzdlGStA3gAqOMKnwr6NH1dEHP GRoTLhW/XmfFRW/jZXkDokzEUMP519sKKaSF6w/CHrPh4QzYORisVXYus+4SE7Inqg5L la+UFr78Q4HIUC/b6bn9I0iDHtnLnBGzqDLKEVFczvEXbvIW4KQC5PZ/yjf83SbSoU4y LOFv5jdle+JYlZSpGrUh1AkeBfH4JHmKGY06YWoCN13eCVr3F+EtbhiZj/f6nqv0xxeY 7Dbw== X-Gm-Message-State: AD7BkJKNs44WYgGzYw1d4YVvjKlGN6541k2jerEE+lEVXBJfDnpUiq26E/Fu4iIzg3hh3w== X-Received: by 10.28.183.84 with SMTP id h81mr3490182wmf.100.1459030597986; Sat, 26 Mar 2016 15:16:37 -0700 (PDT) Received: from ?IPv6:2a02:810a:13c0:9614:d9f:d30d:641:8ce3? ([2a02:810a:13c0:9614:d9f:d30d:641:8ce3]) by smtp.googlemail.com with ESMTPSA id 73sm3190341wmy.22.2016.03.26.15.16.36 for (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Mar 2016 15:16:37 -0700 (PDT) From: Lydia Ickler Content-Type: multipart/alternative; boundary="Apple-Mail=_BA7CBEAC-7DD6-4CD7-A6D0-2DB8E059BB44" Subject: for loop slow Message-Id: Date: Sat, 26 Mar 2016 23:16:36 +0100 To: user@flink.apache.org Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) X-Mailer: Apple Mail (2.3112) --Apple-Mail=_BA7CBEAC-7DD6-4CD7-A6D0-2DB8E059BB44 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, I have an issue with a for-loop. If I set the maximal iteration number i to more than 3 it gets stuck and = I cannot figure out why. With 1, 2 or 3 it runs smoothly. I attached the code below and marked the loop with //PROBLEM. Thanks in advance! Lydia package org.apache.flink.contrib.lifescience.examples; import edu.princeton.cs.algs4.Graph; import edu.princeton.cs.algs4.SymbolDigraph; import org.apache.flink.api.common.functions.FilterFunction; import org.apache.flink.api.common.functions.FlatJoinFunction; import org.apache.flink.api.common.functions.MapFunction; import org.apache.flink.api.java.DataSet; import org.apache.flink.api.java.ExecutionEnvironment; import org.apache.flink.api.java.aggregation.Aggregations; import org.apache.flink.api.java.io.CsvReader; import org.apache.flink.api.java.operators.DataSource; import org.apache.flink.api.java.operators.IterativeDataSet; import org.apache.flink.api.java.tuple.Tuple2; import org.apache.flink.api.java.tuple.Tuple3; import org.apache.flink.contrib.lifescience.networks.algos.DataSetUtils; import = org.apache.flink.contrib.lifescience.networks.datatypes.networks.Network; import = org.apache.flink.contrib.lifescience.networks.datatypes.networks.NetworkEd= ge; import = org.apache.flink.contrib.lifescience.networks.datatypes.networks.NetworkNo= de; import org.apache.flink.core.fs.FileSystem; import org.apache.flink.util.Collector; import java.util.*; import static edu.princeton.cs.algs4.GraphGenerator.simple; public class PowerIteration { //path to input static String input =3D null; //path to output static String output =3D null; //number of iterations (default =3D 7) static int iterations =3D 7; //threshold static double delta =3D 0.01; public void run() throws Exception { ExecutionEnvironment env =3D = ExecutionEnvironment.getExecutionEnvironment(); //read input file DataSet> matrixA =3D = readMatrix(env, input); DataSet> eigenVector; DataSet> eigenValue; //initial: //Approximate EigenVector by PowerIteration eigenVector =3D PowerIteration_getEigenVector(matrixA); //Approximate EigenValue by PowerIteration eigenValue =3D = PowerIteration_getEigenValue(matrixA,eigenVector); //Deflate original matrix matrixA =3D = PowerIteration_getNextMatrix(matrixA,eigenVector,eigenValue); MyResult initial =3D new = MyResult(eigenVector,eigenValue,matrixA); MyResult next =3D null; //PROBLEM!!! get i eigenvalue gaps for(int i=3D0;i<2;i++){ next =3D PowerIteration_routine(initial); initial =3D next; next.gap.print(); } env.execute("Power Iteration"); } public static DataSource> = readMatrix(ExecutionEnvironment env, = String filePath) { CsvReader csvReader =3D env.readCsvFile(filePath); csvReader.fieldDelimiter(","); csvReader.includeFields("ttt"); return csvReader.types(Integer.class, Integer.class, = Double.class); } public static final class ProjectJoinResultMapper implements MapFunction, Tuple3>, Tuple3> { @Override public Tuple3 map( Tuple2, Tuple3> value) throws Exception { Integer row =3D value.f0.f0; Integer column =3D value.f1.f1; Double product =3D value.f0.f2 * value.f1.f2; return new Tuple3(row, column, = product); } } public static final class RQ implements MapFunction, = Tuple3>, Tuple3> { @Override public Tuple3 map( Tuple2, Tuple3> value) throws Exception { return new Tuple3(value.f0.f0,value.f0.f1,value.f0.f2/value.f1.f2); } } public static void main(String[] args) throws Exception { if(args.length<2 || args.length > 4){ System.err.println("Usage: PowerIteration = optional: "); System.exit(0); } input =3D args[0]; output =3D args[1]; if(args.length=3D=3D3) { iterations =3D Integer.parseInt(args[2]); } if(args.length=3D=3D4){ delta =3D Double.parseDouble(args[3]); } new PowerIteration2().run(); } public static final class deltaFilter implements = FlatJoinFunction,Tuple3,Tuple3> { public void join(Tuple3 candidate, = Tuple3 old, Collector> out) { if(!(candidate.f2 =3D=3D old.f2)){ out.collect(candidate); } //if(Math.abs(candidate.f2-old.f2) > delta){ // out.collect(candidate); //} } } public static final class normalizeByMax implements MapFunction, = Tuple3>, Tuple3> { public Tuple3 map( Tuple2, Tuple3> value) throws Exception { return new Tuple3(value.f0.f0,value.f0.f1,value.f0.f2/(value.f1.f2)); } } public static final class firstX implements MapFunction, = Tuple3>, Tuple3> { public Tuple3 map( Tuple2, Tuple3> value) throws Exception { return new Tuple3(value.f0.f0,value.f0.f1,1/(value.f0.f2*value.f1.f2)); } } public static final class resetIndex implements MapFunction, Tuple3> { public Tuple3 map( Tuple3value) throws Exception { return new Tuple3(0,value.f1,value.f2); } } public static final class decBy1 implements MapFunction, Tuple3> { public Tuple3 map( Tuple3value) throws Exception { return new Tuple3(value.f0-1,value.f1-1,value.f2); } } public static final class resetIndex2 implements MapFunction, Tuple3> { public Tuple3 map( Tuple3value) throws Exception { return new Tuple3(value.f0,0,value.f2); } } public static final class MatrixTimesValue implements MapFunction, = Tuple3>, Tuple3> { @Override public Tuple3 map( Tuple2, Tuple3> value) throws Exception { return new Tuple3(value.f0.f0,value.f0.f1,value.f0.f2*(value.f1.f2)); } } public static final class MatrixMinusMatrix implements MapFunction, Tuple3>, Tuple3> { @Override public Tuple3 map( Tuple2, Tuple3> value) throws Exception { Integer row =3D value.f0.f0; Integer column =3D value.f0.f1; Double result =3D value.f0.f2 - value.f1.f2; return new Tuple3(row, column, = result); } } public static final class getGapCenter implements MapFunction, Tuple3>, Tuple3> { @Override public Tuple3 map( Tuple2, Tuple3> value) throws Exception { Integer row =3D value.f0.f0; Integer column =3D value.f0.f1; Double result =3D value.f0.f2 + (2/(Math.abs(value.f1.f2))); return new Tuple3(row, column, = result); } } public static DataSet> = PowerIteration_getEigenVector(DataSet> = matrixA) throws Exception { //get initial vector - which equals matrixA * [1, ... , 1] DataSet> initial0 =3D = matrixA.groupBy(0).aggregate(Aggregations.SUM,2); //normalize by maximum value DataSet> initial=3D = initial0.cross(initial0.maxBy(2)).map(new normalizeByMax()); //BulkIteration to find dominant eigenvector IterativeDataSet> iteration =3D = initial.iterate(iterations); DataSet> intermediate =3D = (matrixA.join(iteration).where(1).equalTo(0) .map(new ProjectJoinResultMapper()).groupBy(0, = 1).aggregate(Aggregations.SUM, = 2)).groupBy(0).aggregate(Aggregations.SUM, 2). cross((matrixA.join(iteration).where(1).equalTo(0) .map(new ProjectJoinResultMapper()).groupBy(0, = 1).aggregate(Aggregations.SUM, = 2)).groupBy(0).aggregate(Aggregations.SUM, 2).maxBy(2)) .map(new normalizeByMax()); DataSet> diffs =3D = iteration.join(intermediate).where(0).equalTo(0).with(new = deltaFilter()); DataSet> eigenVector =3D = iteration.closeWith(intermediate,diffs); return eigenVector; } public static DataSet> = PowerIteration_getEigenValue(DataSet> = matrixA, DataSet> eigenVector) { //determine now EigenValue by approximating the Rayleigh = Quotient: //get Ax DataSet> Ax =3D = matrixA.join(eigenVector).where(1).equalTo(0) .map(new ProjectJoinResultMapper()).groupBy(0, = 1).aggregate(Aggregations.SUM, 2).groupBy(0).aggregate(Aggregations.SUM, = 2); //get Ax * x DataSet> Axx =3D = eigenVector.join(Ax).where(0).equalTo(0) .map(new ProjectJoinResultMapper()).groupBy(0, = 1).aggregate(Aggregations.SUM, 2).aggregate(Aggregations.SUM,2); //now x * x DataSet> xx =3D = eigenVector.join(eigenVector).where(0).equalTo(0) .map(new ProjectJoinResultMapper()).groupBy(0, = 1).aggregate(Aggregations.SUM, 2).aggregate(Aggregations.SUM,2); return Axx.cross(xx).map(new RQ()).aggregate(Aggregations.SUM, = 2); } public static DataSet> = PowerIteration_getNextMatrix(DataSet> = matrixA, DataSet> eigenVector, = DataSet> eigenValue) { DataSet> eigenValueReset =3D = eigenValue.map(new resetIndex()); DataSet> firstVal =3D = eigenVector.filter(new FilterFunction>() { public boolean filter(Tuple3 = value) { return value.f0 =3D=3D 0; } }); firstVal =3D eigenValueReset.cross(firstVal).map(new firstX()); DataSet> firstRow =3D = matrixA.filter(new FilterFunction>() { public boolean filter(Tuple3 = value) { return value.f0 =3D=3D 0; } }); DataSet> x =3D = ((firstRow.map(new = DataSetUtils.transpose())).join(firstVal).where(1).equalTo(0).map(new = MatrixTimesValue())).map(new DataSetUtils.transpose()); DataSet> C =3D = eigenVector.cross(eigenValueReset).map(new MatrixTimesValue()).map(new = resetIndex2()).join(x).where(1).equalTo(0). map(new ProjectJoinResultMapper()).groupBy(0, = 1).aggregate(Aggregations.SUM, 2); matrixA =3D matrixA.join(C).where(0,1).equalTo(0,1).map(new = MatrixMinusMatrix()); matrixA =3D matrixA.filter(new FilterFunction>() { public boolean filter(Tuple3 = value) { return (value.f0 !=3D 0) && (value.f1 !=3D 0); } }); return matrixA.map(new decBy1()); } public MyResult PowerIteration_routine(MyResult initial) throws = Exception { //Approximate EigenVector by PowerIteration DataSet> eigenVector =3D = PowerIteration_getEigenVector(initial.matrixA); //Approximate EigenValue by PowerIteration DataSet> eigenValue =3D = PowerIteration_getEigenValue(initial.matrixA, eigenVector); //get gap DataSet> gap =3D = initial.eigenValue.cross(eigenValue).map(new MatrixMinusMatrix()); //Deflate original matrix DataSet> matrixA =3D = PowerIteration_getNextMatrix(initial.matrixA,eigenVector,eigenValue); return new MyResult(eigenVector,eigenValue,matrixA,gap); } public class MyResult { DataSet> eigenVector; DataSet> eigenValue; DataSet> gap; DataSet> matrixA; public MyResult(DataSet> = eigenVector, DataSet> = eigenValue,DataSet> matrixA){ this.eigenVector =3D eigenVector; this.eigenValue =3DeigenValue; this.matrixA =3D matrixA; } public MyResult(DataSet> = eigenVector, DataSet> = eigenValue,DataSet> matrixA, = DataSet> gap){ this.eigenVector =3D eigenVector; this.eigenValue =3DeigenValue; this.matrixA =3D matrixA; this.gap =3D gap; } } } --Apple-Mail=_BA7CBEAC-7DD6-4CD7-A6D0-2DB8E059BB44 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi,

I = have an issue with a for-loop.
If I set the maximal = iteration number i to more than 3 it gets stuck and I cannot figure out = why.
With 1, 2 or 3 it runs smoothly.
I attached the code below and marked the loop with = //PROBLEM.

Thanks in advance!
Lydia

package =
org.apache.flink.contrib.lifescience.examples;

import edu.princeton.cs.algs4.Graph;
import =
edu.princeton.cs.algs4.SymbolDigraph;
import =
org.apache.flink.api.common.functions.FilterFunction;
import =
org.apache.flink.api.common.functions.FlatJoinFunction;
import =
org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.java.DataSet;
import =
org.apache.flink.api.java.ExecutionEnvironment;
import =
org.apache.flink.api.java.aggregation.Aggregations;
import org.apache.flink.api.java.io.CsvReader;
import =
org.apache.flink.api.java.operators.DataSource;
import =
org.apache.flink.api.java.operators.IterativeDataSet;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.tuple.Tuple3;
import =
org.apache.flink.contrib.lifescience.networks.algos.DataSetUtils;import =
org.apache.flink.contrib.lifescience.networks.datatypes.networks.Ne=
twork;
import =
org.apache.flink.contrib.lifescience.networks.datatypes.networks.Ne=
tworkEdge;
import =
org.apache.flink.contrib.lifescience.networks.datatypes.networks.Ne=
tworkNode;
import org.apache.flink.core.fs.FileSystem;
import org.apache.flink.util.Collector;

import java.util.*;

import static =
edu.princeton.cs.algs4.GraphGenerator.simple;

public class PowerIteration {

    //path to input
    static =
String input =3D null;
    //path to output
    static =
String output =3D null;
    //number of iterations (default =3D 7)
    static int iterations =
=3D 7;
    //threshold
    static double =
delta =
=3D 0.01;

    public void =
run() throws Exception {
        =
ExecutionEnvironment env =3D ExecutionEnvironment.getExecutionEnvironment();

=
        //read=
 input file
        =
DataSet<Tuple3<Integer, Integer, Double>> matrixA =3D =
readMatrix(env, =
input);

        =
DataSet<Tuple3<Integer, Integer, Double>> eigenVector;
        DataSet<Tuple3<Integer, Integer, Double>> =
eigenValue;

        //initial:
        //Approximate EigenVector by PowerIteration
        eigenVector =3D PowerIteration_getEigenVector(matrixA);
 =
       //Approximate EigenValue by PowerIteration
        eigenValue =3D PowerIteration_getEigenValue(matrixA,eigenVector);
        //Deflate original matrix
        =
matrixA =3D PowerIteration_getNextMatrix(matrixA,eigenVector,eigenVa=
lue);

        MyResult initial =3D new =
MyResult(eigenVector,eigenValue,matrixA);

        MyResult next =3D null;

        //PROBLEM!!! get i =
eigenvalue gaps
        =
for(int i=3D0;i<2;i++){
            next =3D =
PowerIteration_routine(initial);
            initial =3D =
next;
            next.gap.print();
        }

        env.execute("Power =
Iteration");
    }

    =
public static =
DataSource<Tuple3<Integer, Integer, Double>> =
readMatrix(ExecutionEnvironment env,
                      =
                                                    String filePath) =
{
        CsvReader csvReader =3D =
env.readCsvFile(filePath);
        =
csvReader.fieldDelimiter(",");
        =
csvReader.includeFields("ttt");
        return =
csvReader.types(Integer.class, =
Integer.class, Double.class);
    }

    public static final =
class ProjectJoinResultMapper implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, =
Double>,
                    Tuple3<Integer, =
Integer, Double>>,
                    =
Tuple3<Integer, Integer, Double>> {
        @Override
        public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>> value)
         =
       throws =
Exception {
            Integer row =3D value.f0.f0;
            Integer column =3D value.f1.f1;
            Double product =3D value.f0.f2 * =
value.f1.f2;
            return new =
Tuple3<Integer, Integer, Double>(row, column, product);
        }
    }

   =
 public =
static final class RQ implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>>,
               =
     Tuple3<Integer, Integer, Double>> {

        @Override
        =
public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>> value)
         =
       throws =
Exception {

            return new =
Tuple3<Integer, Integer, Double>(value.f0.f0,value.f0.f1,value.f0.f2/value.f1.f2);
        }
    }

   =
 public =
static void main(String[] args) throws =
Exception {
        if(args.length<2 =
|| args.length > 4){
            System.err.println("Usage: =
PowerIteration <input path> <result path> optional: =
<iterations> <threshold diff>");
       =
     System.exit(0);
        }

        input =3D args[0];
        output =3D =
args[1];

        if(args.length=3D=3D3) {
       =
     iterations =3D Integer.parseInt(args[2]);
        =
}
        if(args.length=3D=3D4){
            delta =3D =
Double.parseDouble(args[3]);
        }

        new PowerIteration2().run();
    }

    public static final =
class deltaFilter implements FlatJoinFunction<Tuple3<Integer, =
Integer, Double>,Tuple3<Integer, Integer, =
Double>,Tuple3<Integer, Integer, Double>> {

        public void join(Tuple3<Integer, Integer, =
Double> candidate, Tuple3<Integer, Integer, Double> old, =
Collector<Tuple3<Integer, Integer, Double>> out) {

            if(!(candidate.f2 =3D=3D =
old.f2)){
                =
out.collect(candidate);
            }

            //if(Math.abs(candidate.f2-old.f2) > delta){
            //    out.collect(candidate);
            //}

        }
    }

    public static final class normalizeByMax implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>>,
               =
     Tuple3<Integer, Integer, Double>> {

        public Tuple3<Integer, Integer, Double> map(
                Tuple2<Tuple3<Integer, Integer, =
Double>, Tuple3<Integer, Integer, Double>> value)
                throws Exception {
            return new =
Tuple3<Integer, Integer, Double>(value.f0.f0,value.f0.f1,value.f0.f2/(value.f1.f2));
        }
    }

   =
 public =
static final class firstX implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>>,
               =
     Tuple3<Integer, Integer, Double>> {

        public Tuple3<Integer, Integer, Double> map(
                Tuple2<Tuple3<Integer, Integer, =
Double>, Tuple3<Integer, Integer, Double>> value)
                throws Exception {
            return new =
Tuple3<Integer, Integer, Double>(value.f0.f0,value.f0.f1,1/(value.f0.f2*value.f1.f2));
        }
    }

   =
 public =
static final class resetIndex implements
            MapFunction<Tuple3<Integer, Integer, =
Double>,
                    Tuple3<Integer, =
Integer, Double>> {

        public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple3<Integer, Integer, Double>value)
      =
          throws Exception {
            return new =
Tuple3<Integer, Integer, Double>(0,value.f1,value.f2);
        }
    }


    public static final class decBy1 implements
            MapFunction<Tuple3<Integer, Integer, =
Double>,
                    Tuple3<Integer, =
Integer, Double>> {

        public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple3<Integer, Integer, Double>value)
      =
          throws Exception {
            return new =
Tuple3<Integer, Integer, Double>(value.f0-1,value.f1-1,value.f2);
        }
    }

   =
 public =
static final class resetIndex2 implements
            MapFunction<Tuple3<Integer, Integer, =
Double>,
                    Tuple3<Integer, =
Integer, Double>> {

        public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple3<Integer, Integer, Double>value)
      =
          throws Exception {
            return new =
Tuple3<Integer, Integer, Double>(value.f0,0,value.f2);
        }
    }

   =
 public =
static final class MatrixTimesValue implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>>,
               =
     Tuple3<Integer, Integer, Double>> {

        @Override
        =
public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>> value)
         =
       throws =
Exception {

            return new =
Tuple3<Integer, Integer, Double>(value.f0.f0,value.f0.f1,value.f0.f2*(value.f1.f2));
        }
    }

   =
 public =
static final class MatrixMinusMatrix implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, =
Double>,
                    Tuple3<Integer, =
Integer, Double>>,
                    =
Tuple3<Integer, Integer, Double>> {
        @Override
        public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>> value)
         =
       throws =
Exception {
            Integer row =3D value.f0.f0;
            Integer column =3D value.f0.f1;
            Double result =3D value.f0.f2 - =
value.f1.f2;
            return new =
Tuple3<Integer, Integer, Double>(row, column, result);
        }
    }

   =
 public =
static final class getGapCenter implements
            =
MapFunction<Tuple2<Tuple3<Integer, Integer, =
Double>,
                    Tuple3<Integer, =
Integer, Double>>,
                    =
Tuple3<Integer, Integer, Double>> {
        @Override
        public =
Tuple3<Integer, Integer, Double> map(
        =
        Tuple2<Tuple3<Integer, Integer, Double>, =
Tuple3<Integer, Integer, Double>> value)
         =
       throws =
Exception {
            Integer row =3D value.f0.f0;
            Integer column =3D value.f0.f1;
            Double result =3D value.f0.f2 + (2/(Math.abs(value.f1.f2)));
            return new Tuple3<Integer, Integer, Double>(row, =
column, result);
        }
    }


    public static =
DataSet<Tuple3<Integer, Integer, Double>> =
PowerIteration_getEigenVector(DataSet<Tuple3<Integer, Integer, =
Double>> matrixA) throws Exception {

       =
 //get =
initial vector - which equals matrixA * [1, ... , 1]
        DataSet<Tuple3<Integer, Integer, =
Double>> initial0 =3D matrixA.groupBy(0).aggregate(Aggregations.SUM,2);

        //normalize by =
maximum value
        =
DataSet<Tuple3<Integer, Integer, Double>> initial=3D =
initial0.cross(initial0.maxBy(2)).map(new normalizeByMax());

   =
     //BulkIteration to find dominant eigenvector
        IterativeDataSet<Tuple3<Integer, =
Integer, Double>> iteration =3D initial.iterate(iterations);

        =
DataSet<Tuple3<Integer, Integer, Double>> intermediate =3D =
(matrixA.join(iteration).where(1).equalTo(0)
                .map(new =
ProjectJoinResultMapper()).groupBy(0, 1).aggregate(Aggregations.SUM, 2)).groupBy(0).aggregate(Aggregations.SUM, 2).
                =
cross((matrixA.join(iteration).where(1).equalTo(0)
                        .map(new =
ProjectJoinResultMapper()).groupBy(0, 1).aggregate(Aggregations.SUM, 2)).groupBy(0).aggregate(Aggregations.SUM, 2).maxBy(2))
                .map(new =
normalizeByMax());

        =
DataSet<Tuple3<Integer, Integer, Double>> diffs =3D =
iteration.join(intermediate).where(0).equalTo(0).with(new deltaFilter());
        =
DataSet<Tuple3<Integer, Integer, Double>> eigenVector  =3D =
iteration.closeWith(intermediate,diffs);

   =
     return =
eigenVector;
    }

    =
public static =
DataSet<Tuple3<Integer, Integer, Double>> =
PowerIteration_getEigenValue(DataSet<Tuple3<Integer, Integer, =
Double>> matrixA, DataSet<Tuple3<Integer, Integer, =
Double>> eigenVector) {

        //determine now =
EigenValue by approximating the Rayleigh Quotient:
        //get Ax
        =
DataSet<Tuple3<Integer, Integer, Double>> Ax =3D =
matrixA.join(eigenVector).where(1).equalTo(0)
                .map(new =
ProjectJoinResultMapper()).groupBy(0, 1).aggregate(Aggregations.SUM, 2).groupBy(0).aggregate(Aggregations.SUM, 2);
        //get Ax * x
        DataSet<Tuple3<Integer, Integer, =
Double>> Axx =3D eigenVector.join(Ax).where(0).equalTo(0)
             =
   .map(new =
ProjectJoinResultMapper()).groupBy(0, 1).aggregate(Aggregations.SUM, 2).aggregate(Aggregations.SUM,2);

        //now x * x
        DataSet<Tuple3<Integer, Integer, =
Double>> xx =3D eigenVector.join(eigenVector).where(0).equalTo(0)
             =
   .map(new =
ProjectJoinResultMapper()).groupBy(0, 1).aggregate(Aggregations.SUM, 2).aggregate(Aggregations.SUM,2);

        return =
Axx.cross(xx).map(new RQ()).aggregate(Aggregations.SUM, 2);
    }

  =
  public =
static DataSet<Tuple3<Integer, Integer, Double>> =
PowerIteration_getNextMatrix(DataSet<Tuple3<Integer, Integer, =
Double>> matrixA, DataSet<Tuple3<Integer, Integer, =
Double>> eigenVector, DataSet<Tuple3<Integer, Integer, =
Double>> eigenValue) {

        =
DataSet<Tuple3<Integer, Integer, Double>> eigenValueReset =3D =
eigenValue.map(new resetIndex());
        =
DataSet<Tuple3<Integer, Integer, Double>> firstVal =3D =
eigenVector.filter(new FilterFunction<Tuple3<Integer, Integer, =
Double>>() {
            public boolean =
filter(Tuple3<Integer, Integer, Double> value) {
                return value.f0 =3D=3D =
0;
       =
     }
        });
        firstVal =3D =
eigenValueReset.cross(firstVal).map(new =
firstX());
        DataSet<Tuple3<Integer, =
Integer, Double>> firstRow =3D matrixA.filter(new =
FilterFunction<Tuple3<Integer, Integer, Double>>() =
{
            public boolean =
filter(Tuple3<Integer, Integer, Double> value) {
                return value.f0 =3D=3D =
0;
       =
     }
        });
        =
DataSet<Tuple3<Integer, Integer, Double>> x =3D =
((firstRow.map(new =
DataSetUtils.transpose())).join(firstVal).where(1).equalTo(0).map(new =
MatrixTimesValue())).map(new =
DataSetUtils.transpose());
        =
DataSet<Tuple3<Integer, Integer, Double>> C =3D =
eigenVector.cross(eigenValueReset).map(new =
MatrixTimesValue()).map(new =
resetIndex2()).join(x).where(1).equalTo(0).
                map(new =
ProjectJoinResultMapper()).groupBy(0, 1).aggregate(Aggregations.SUM, 2);
        matrixA =3D =
matrixA.join(C).where(0,1).equalTo(0,1).map(new MatrixMinusMatrix());
        =
matrixA =3D matrixA.filter(new FilterFunction<Tuple3<Integer, Integer, =
Double>>() {
            public boolean =
filter(Tuple3<Integer, Integer, Double> value) {
                return (value.f0 !=3D =
0) && =
(value.f1 =
!=3D 0);
            }
        });

        return matrixA.map(new =
decBy1());
    }

    =
public =
MyResult PowerIteration_routine(MyResult initial) throws =
Exception {

        //Approximate =
EigenVector by PowerIteration
        =
DataSet<Tuple3<Integer, Integer, Double>> eigenVector =
=3D PowerIteration_getEigenVector(initial.matrixA);
        //Approximate EigenValue by PowerIteration
        DataSet<Tuple3<Integer, Integer, =
Double>> eigenValue =3D PowerIteration_getEigenValue(initial.matrixA, =
eigenVector);
        //get gap
        DataSet<Tuple3<Integer, Integer, =
Double>> gap =3D initial.eigenValue.cross(eigenValue).map(new =
MatrixMinusMatrix());
        //Deflate original =
matrix
        =
DataSet<Tuple3<Integer, Integer, Double>> matrixA =3D =
PowerIteration_getNextMatrix(initial.matrixA,eigenVector,eigenValue);

        return new =
MyResult(eigenVector,eigenValue,matrixA,gap);
    =
}

    public class =
MyResult {
        DataSet<Tuple3<Integer, =
Integer, Double>> eigenVector;
        =
DataSet<Tuple3<Integer, Integer, Double>> eigenValue;        DataSet<Tuple3<Integer, Integer, =
Double>> gap;
        =
DataSet<Tuple3<Integer, Integer, Double>> matrixA;

        public =
MyResult(DataSet<Tuple3<Integer, Integer, Double>> =
eigenVector, DataSet<Tuple3<Integer, Integer, Double>> =
eigenValue,DataSet<Tuple3<Integer, Integer, Double>> =
matrixA){
            this.eigenVector =
=3D eigenVector;
            this.eigenValue =
=3DeigenValue;
            this.matrixA =3D =
matrixA;
        }

        =
public =
MyResult(DataSet<Tuple3<Integer, Integer, Double>> =
eigenVector, DataSet<Tuple3<Integer, Integer, Double>> =
eigenValue,DataSet<Tuple3<Integer, Integer, Double>> =
matrixA, DataSet<Tuple3<Integer, Integer, Double>> gap){
            this.eigenVector =3D eigenVector;
            =
this.eigenValue =3DeigenValue;
            =
this.matrixA =3D matrixA;
            this.gap =3D =
gap;
        }
    }

}

= --Apple-Mail=_BA7CBEAC-7DD6-4CD7-A6D0-2DB8E059BB44--