mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Palleti, Pallavi" <pallavi.pall...@corp.aol.com>
Subject RE: [jira] Commented: (MAHOUT-79) Improving the speed of Fuzzy K-Means by optimizing data transfer between map and reduce tasks
Date Sat, 18 Oct 2008 06:42:25 GMT
Hi Grant,
 Let me know if you are still facing this issue?

Thanks
Pallavi

-----Original Message-----
From: Grant Ingersoll (JIRA) [mailto:jira@apache.org] 
Sent: Friday, October 17, 2008 11:48 PM
To: mahout-dev@lucene.apache.org
Subject: [jira] Commented: (MAHOUT-79) Improving the speed of Fuzzy K-Means by optimizing
data transfer between map and reduce tasks


    [ https://issues.apache.org/jira/browse/MAHOUT-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640608#action_12640608
] 

Grant Ingersoll commented on MAHOUT-79:
---------------------------------------

Pallavi,

I'm getting:
 [junit] 08/10/17 14:15:03 INFO mapred.FileInputFormat: Total input paths to process : 2
    [junit] 08/10/17 14:15:03 INFO mapred.FileInputFormat: Total input paths to process :
2
    [junit] 08/10/17 14:15:03 INFO mapred.JobClient: Running job: job_local_0002
    [junit] 08/10/17 14:15:03 INFO mapred.FileInputFormat: Total input paths to process :
2
    [junit] 08/10/17 14:15:03 INFO mapred.FileInputFormat: Total input paths to process :
2
    [junit] 08/10/17 14:15:03 INFO mapred.MapTask: numReduceTasks: 0
    [junit] 08/10/17 14:15:03 INFO fuzzykmeans.FuzzyKMeansMapper: In Mapper Configure:
    [junit] 08/10/17 14:15:03 WARN mapred.LocalJobRunner: job_local_0002
    [junit] java.lang.NullPointerException: Cluster is empty!!!
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansMapper.configure(FuzzyKMeansMapper.java:76)
    [junit]     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    [junit]     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
    [junit]     at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:33)
    [junit]     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
    [junit]     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
    [junit]     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:223)
    [junit]     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:157)
    [junit] 08/10/17 14:15:04 WARN fuzzykmeans.FuzzyKMeansDriver: java.io.IOException: Job
failed!
    [junit] java.io.IOException: Job failed!
    [junit]     at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1113)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansDriver.runClustering(FuzzyKMeansDriver.java:207)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansDriver.runJob(FuzzyKMeansDriver.java:116)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering.testFuzzyKMeansMRJob(TestFuzzyKmeansClustering.java:248)
    [junit]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [junit]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [junit]     at java.lang.reflect.Method.invoke(Method.java:597)
    [junit]     at junit.framework.TestCase.runTest(TestCase.java:164)
    [junit]     at junit.framework.TestCase.runBare(TestCase.java:130)
    [junit]     at junit.framework.TestResult$1.protect(TestResult.java:106)
    [junit]     at junit.framework.TestResult.runProtected(TestResult.java:124)
    [junit]     at junit.framework.TestResult.run(TestResult.java:109)
    [junit]     at junit.framework.TestCase.run(TestCase.java:120)
    [junit]     at junit.framework.TestSuite.runTest(TestSuite.java:230)
    [junit]     at junit.framework.TestSuite.run(TestSuite.java:225)
    [junit]     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421)
    [junit]     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:912)
    [junit]     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:766)
    [junit] ------------- ---------------- ---------------
    [junit] Testcase: testFuzzyKMeansMRJob(org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering):
Caused an ERROR
    [junit] output/points/part-00000 (No such file or directory)
    [junit] java.io.FileNotFoundException: output/points/part-00000 (No such file or directory)
    [junit]     at java.io.FileInputStream.open(Native Method)
    [junit]     at java.io.FileInputStream.<init>(FileInputStream.java:106)
    [junit]     at java.io.FileInputStream.<init>(FileInputStream.java:66)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering.testFuzzyKMeansMRJob(TestFuzzyKmeansClustering.java:257)
    [junit] 
    [junit] 
    [junit] Testcase: testFuzzyKMeansReducer(org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering):
      Caused an ERROR
    [junit] For input string: "9.0, [s2, 0"
    [junit] java.lang.NumberFormatException: For input string: "9.0, [s2, 0"
    [junit]     at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
    [junit]     at java.lang.Double.valueOf(Double.java:475)
    [junit]     at java.lang.Double.<init>(Double.java:567)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansReducer.reduce(FuzzyKMeansReducer.java:58)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering.testFuzzyKMeansReducer(TestFuzzyKmeansClustering.java:452)
    [junit] 
    [junit] 
    [junit] Testcase: testFuzzyKMeansClusterMapper(org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering):
Caused an ERROR
    [junit] For input string: "9.0, [s2, 0"
    [junit] java.lang.NumberFormatException: For input string: "9.0, [s2, 0"
    [junit]     at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
    [junit]     at java.lang.Double.valueOf(Double.java:475)
    [junit]     at java.lang.Double.<init>(Double.java:567)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.FuzzyKMeansReducer.reduce(FuzzyKMeansReducer.java:58)
    [junit]     at org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering.testFuzzyKMeansClusterMapper(TestFuzzyKmeansClustering.java:528)
    [junit] 
    [junit] 
    [junit] Test org.apache.mahout.clustering.fuzzykmeans.TestFuzzyKmeansClustering FAILED
[junitreport] Processing /Volumes/User/grantingersoll/projects/lucene/mahout/mahout-clean/core/build/test-results/TESTS-TestSuites.xml
to /tmp/null901114900
[junitreport] Loading stylesheet jar:file:/usr/local/ant/lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl
[junitreport] Transform time: 589ms
[junitreport] Deleting: /tmp/null901114900

BUILD FAILED
/Volumes/User/grantingersoll/projects/lucene/mahout/mahout-clean/core/build.xml:229: Tests
failed!



> Improving the speed of Fuzzy K-Means by optimizing data transfer between map and reduce
tasks
> ---------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-79
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-79
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Clustering
>            Reporter: Pallavi Palleti
>            Assignee: Grant Ingersoll
>             Fix For: 0.1
>
>         Attachments: FUZZY-79.patch, FUZZY-79.patch, FUZZY.patch
>
>
> Improve the speed of fuzzy k-Means by passing only the cluster-id info as key output
of mapper task and reading the cluster information in reducer task where this info is needed.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Mime
View raw message