Return-Path: Delivered-To: apmail-lucene-mahout-commits-archive@minotaur.apache.org Received: (qmail 3963 invoked from network); 11 Aug 2009 12:05:09 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 11 Aug 2009 12:05:09 -0000 Received: (qmail 39311 invoked by uid 500); 11 Aug 2009 12:05:16 -0000 Delivered-To: apmail-lucene-mahout-commits-archive@lucene.apache.org Received: (qmail 39232 invoked by uid 500); 11 Aug 2009 12:05:16 -0000 Mailing-List: contact mahout-commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mahout-dev@lucene.apache.org Delivered-To: mailing list mahout-commits@lucene.apache.org Received: (qmail 39223 invoked by uid 99); 11 Aug 2009 12:05:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Aug 2009 12:05:15 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Aug 2009 12:05:05 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id ABFD6238885B; Tue, 11 Aug 2009 12:04:45 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r803081 [1/7] - in /lucene/mahout/trunk: core/src/main/java/org/apache/mahout/cf/taste/eval/ core/src/main/java/org/apache/mahout/cf/taste/hadoop/ core/src/main/java/org/apache/mahout/cf/taste/impl/common/ core/src/main/java/org/apache/maho... Date: Tue, 11 Aug 2009 12:04:38 -0000 To: mahout-commits@lucene.apache.org From: srowen@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20090811120445.ABFD6238885B@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: srowen Date: Tue Aug 11 12:04:35 2009 New Revision: 803081 URL: http://svn.apache.org/viewvc?rev=803081&view=rev Log: MAHOUT-158 Added: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/DataModelBuilder.java - copied, changed from r802574, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderBuilder.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastByIDMap.java - copied, changed from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastIDSet.java - copied, changed from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastSet.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPair.java - copied, changed from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveArrayIterator.java - copied, changed from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/ArrayIterator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveIterator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/SamplingLongPrimitiveIterator.java - copied, changed from r802574, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/SamplingIterator.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/common/FastIDSetTest.java (contents, props changed) - copied, changed from r802574, lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/common/FastSetTest.java Removed: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastSet.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/common/FastSetTest.java Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderIRStatsEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ByItemIDComparator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemItemWritable.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemPrefWritable.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommendedItemsWritable.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderJob.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderMapper.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsMapper.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsReducer.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/IteratorUtils.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RandomUtils.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RefreshHelper.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/eval/AbstractDifferenceRecommenderEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/eval/AverageAbsoluteDifferenceRecommenderEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/eval/GenericRecommenderIRStatsEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/eval/LoadEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/eval/RMSRecommenderEvaluator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/BooleanItemPreferenceArray.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/BooleanPreference.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/BooleanUserPreferenceArray.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/GenericBooleanPrefDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/GenericDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/GenericItemPreferenceArray.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/GenericPreference.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/GenericUserPreferenceArray.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/file/FileDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/jdbc/AbstractBooleanPrefJDBCDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/jdbc/AbstractJDBCDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/jdbc/MySQLBooleanPrefJDBCDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/model/jdbc/MySQLJDBCDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/neighborhood/CachingUserNeighborhood.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/neighborhood/NearestNUserNeighborhood.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/neighborhood/ThresholdUserNeighborhood.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/AbstractRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/ByRescoreComparator.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/CachingRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/ClusterSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/FarthestNeighborClusterSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericItemBasedRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericRecommendedItem.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericUserBasedRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/ItemAverageRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/ItemUserAverageRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/NearestNeighborClusterSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/NullRescorer.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/SimilarUser.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/TopItems.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/TreeClusteringRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/TreeClusteringRecommender2.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/knn/KnnItemBasedRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/slopeone/MemoryDiffStorage.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/slopeone/SlopeOneRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/slopeone/jdbc/AbstractJDBCDiffStorage.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/slopeone/jdbc/MySQLJDBCDiffStorage.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/recommender/svd/SVDRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/AbstractSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/AveragingPreferenceInferrer.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/CachingItemSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/CachingUserSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/GenericItemSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/GenericUserSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/LogLikelihoodSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/SpearmanCorrelationSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/TanimotoCoefficientSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/jdbc/AbstractJDBCItemSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/jdbc/MySQLJDBCItemSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/transforms/CaseAmplification.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/transforms/Counters.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/transforms/InverseUserFrequency.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/transforms/ZScore.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/model/DataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/model/JDBCDataModel.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/model/Preference.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/model/PreferenceArray.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/neighborhood/UserNeighborhood.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/recommender/ClusteringRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/recommender/ItemBasedRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/recommender/RecommendedItem.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/recommender/Recommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/recommender/UserBasedRecommender.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/recommender/slopeone/DiffStorage.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/ItemSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/PreferenceInferrer.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/UserSimilarity.java lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/transforms/SimilarityTransform.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/LoadTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/TasteTestCase.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/eval/AverageAbsoluteDifferenceRecommenderEvaluatorTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/eval/GenericRecommenderIRStatsEvaluatorImplTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/eval/RMSRecommenderEvaluatorTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/model/file/FileDataModelTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/neighborhood/DummySimilarity.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/neighborhood/NearestNNeighborhoodTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/neighborhood/ThresholdNeighborhoodTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/CachingRecommenderTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/GenericItemBasedRecommenderTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/GenericUserBasedRecommenderTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/MockRecommender.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/NullRescorerTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/TreeClusteringRecommenderTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/slopeone/MemoryDiffStorageTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/recommender/slopeone/SlopeOneRecommenderTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/AveragingPreferenceInferrerTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/EuclideanDistanceSimilarityTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/GenericItemSimilarityTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/LogLikelihoodSimilarityTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/PearsonCorrelationSimilarityTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/SimilarityTestCase.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/SpearmanCorrelationSimilarityTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/similarity/TanimotoCoefficientSimilarityTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/transforms/CaseAmplificationTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/transforms/InverseUserFrequencyTest.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/transforms/TransformTestCase.java lucene/mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/impl/transforms/ZScoreTest.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/ejb/RecommenderEJB.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/ejb/RecommenderEJBBean.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/ejb/RecommenderEJBLocal.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/bookcrossing/BookCrossingRecommender.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/bookcrossing/BookCrossingRecommenderEvaluatorRunner.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/grouplens/GroupLensRecommender.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/grouplens/GroupLensRecommenderEvaluatorRunner.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/jester/JesterDataModel.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/jester/JesterRecommender.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/jester/JesterRecommenderEvaluatorRunner.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/netflix/NetflixDataModel.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/netflix/NetflixFileDataModel.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/netflix/NetflixRecommender.java lucene/mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/example/netflix/NetflixRecommenderEvaluatorRunner.java lucene/mahout/trunk/taste-web/src/main/java/org/apache/mahout/cf/taste/web/RecommenderServlet.java Copied: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/DataModelBuilder.java (from r802574, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderBuilder.java) URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/DataModelBuilder.java?p2=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/DataModelBuilder.java&p1=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderBuilder.java&r1=802574&r2=803081&rev=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderBuilder.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/DataModelBuilder.java Tue Aug 11 12:04:35 2009 @@ -18,22 +18,27 @@ package org.apache.mahout.cf.taste.eval; import org.apache.mahout.cf.taste.common.TasteException; +import org.apache.mahout.cf.taste.impl.common.FastByIDMap; import org.apache.mahout.cf.taste.model.DataModel; +import org.apache.mahout.cf.taste.model.PreferenceArray; import org.apache.mahout.cf.taste.recommender.Recommender; /** - *

Implementations of this inner interface are simple helper classes which create a {@link Recommender} to be - * evaluated based on the given {@link DataModel}.

+ *

Implementations of this inner interface are simple helper classes which create a {@link DataModel} to be + * used while evaluating a {@link Recommender}. + * + * @see RecommenderBuilder + * @see RecommenderEvaluator */ -public interface RecommenderBuilder { +public interface DataModelBuilder { /** - *

Builds a {@link Recommender} implementation to be evaluated, using the given {@link DataModel}.

+ *

Builds a {@link DataModel} implementation to be used in an evaluation, given training data.

* - * @param dataModel {@link DataModel} to build the {@link Recommender} on - * @return {@link Recommender} based upon the given {@link DataModel} + * @param trainingData data to be used in the {@link DataModel} + * @return {@link DataModel} based upon the given data * @throws TasteException if an error occurs while accessing the {@link DataModel} */ - Recommender buildRecommender(DataModel dataModel) throws TasteException; + DataModel buildDataModel(FastByIDMap trainingData) throws TasteException; -} +} \ No newline at end of file Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderEvaluator.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderEvaluator.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderEvaluator.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderEvaluator.java Tue Aug 11 12:04:35 2009 @@ -21,8 +21,8 @@ import org.apache.mahout.cf.taste.model.DataModel; /** - *

Implementations of this interface evaluate the quality of a {@link org.apache.mahout.cf.taste.recommender.Recommender}'s - * recommendations.

+ *

Implementations of this interface evaluate the quality of a + * {@link org.apache.mahout.cf.taste.recommender.Recommender}'s recommendations.

*/ public interface RecommenderEvaluator { @@ -49,9 +49,12 @@ * * @param recommenderBuilder object that can build a {@link org.apache.mahout.cf.taste.recommender.Recommender} to * test + * @param dataModelBuilder @param dataModelBuilder {@link DataModelBuilder} to use, or if null, a default {@link DataModel} implementation + * will be used * @param dataModel dataset to test on * @param trainingPercentage percentage of each user's preferences to use to produce recommendations; the rest are - * compared to estimated preference values to evaluate {@link org.apache.mahout.cf.taste.recommender.Recommender} + * compared to estimated preference values to evaluate + * {@link org.apache.mahout.cf.taste.recommender.Recommender} * performance * @param evaluationPercentage percentage of users to use in evaluation * @return a "score" representing how well the {@link org.apache.mahout.cf.taste.recommender.Recommender}'s estimated @@ -59,6 +62,7 @@ * @throws TasteException if an error occurs while accessing the {@link DataModel} */ double evaluate(RecommenderBuilder recommenderBuilder, + DataModelBuilder dataModelBuilder, DataModel dataModel, double trainingPercentage, double evaluationPercentage) throws TasteException; Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderIRStatsEvaluator.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderIRStatsEvaluator.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderIRStatsEvaluator.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/eval/RecommenderIRStatsEvaluator.java Tue Aug 11 12:04:35 2009 @@ -32,6 +32,8 @@ /** * @param recommenderBuilder object that can build a {@link org.apache.mahout.cf.taste.recommender.Recommender} to * test + * @param dataModelBuilder {@link DataModelBuilder} to use, or if null, a default {@link DataModel} implementation + * will be used * @param dataModel dataset to test on * @param rescorer if any, to use when computing recommendations * @param at as in, "precision at 5". The number of recommendations to consider when evaluating @@ -42,8 +44,9 @@ * @throws TasteException if an error occurs while accessing the {@link DataModel} */ IRStatistics evaluate(RecommenderBuilder recommenderBuilder, + DataModelBuilder dataModelBuilder, DataModel dataModel, - Rescorer> rescorer, + Rescorer rescorer, int at, double relevanceThreshold, double evaluationPercentage) throws TasteException; Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ByItemIDComparator.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ByItemIDComparator.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ByItemIDComparator.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ByItemIDComparator.java Tue Aug 11 12:04:35 2009 @@ -30,7 +30,9 @@ @Override public int compare(ItemPrefWritable a, ItemPrefWritable b) { - return a.getItemID().compareTo(b.getItemID()); + long idA = a.getItemID(); + long idB = b.getItemID(); + return idA < idB ? -1 : idA > idB ? 1 : 0; } } Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemItemWritable.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemItemWritable.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemItemWritable.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemItemWritable.java Tue Aug 11 12:04:35 2009 @@ -18,6 +18,7 @@ package org.apache.mahout.cf.taste.hadoop; import org.apache.hadoop.io.WritableComparable; +import org.apache.mahout.cf.taste.impl.common.RandomUtils; import java.io.DataInput; import java.io.DataOutput; @@ -26,36 +27,36 @@ /** A {@link WritableComparable} encapsulating two items. */ public final class ItemItemWritable implements WritableComparable { - private String itemAID; - private String itemBID; + private long itemAID; + private long itemBID; public ItemItemWritable() { // do nothing } - public ItemItemWritable(String itemAID, String itemBID) { + public ItemItemWritable(long itemAID, long itemBID) { this.itemAID = itemAID; this.itemBID = itemBID; } - public String getItemAID() { + public long getItemAID() { return itemAID; } - public String getItemBID() { + public long getItemBID() { return itemBID; } @Override public void write(DataOutput out) throws IOException { - out.writeUTF(itemAID); - out.writeUTF(itemBID); + out.writeLong(itemAID); + out.writeLong(itemBID); } @Override public void readFields(DataInput in) throws IOException { - itemAID = in.readUTF(); - itemBID = in.readUTF(); + itemAID = in.readLong(); + itemBID = in.readLong(); } public static ItemItemWritable read(DataInput in) throws IOException { @@ -69,27 +70,32 @@ if (this == that) { return 0; } - int compare = itemAID.compareTo(that.itemAID); - return compare == 0 ? itemBID.compareTo(that.itemBID) : compare; + if (itemAID < that.itemAID) { + return -1; + } else if (itemAID > that.itemAID) { + return 1; + } else { + return itemBID < that.itemBID ? -1 : itemBID > that.itemBID ? 1 : 0; + } } @Override public int hashCode() { - return itemAID.hashCode() + 31 * itemBID.hashCode(); + return RandomUtils.hashLong(itemAID) + 31 * RandomUtils.hashLong(itemBID); } @Override public boolean equals(Object o) { if (o instanceof ItemItemWritable) { ItemItemWritable that = (ItemItemWritable) o; - return this == that || (itemAID.equals(that.itemAID) && itemBID.equals(that.itemBID)); + return itemAID == that.itemAID && itemBID == that.itemBID; } return false; } @Override public String toString() { - return itemAID + '\t' + itemBID; + return itemAID + "\t" + itemBID; } } \ No newline at end of file Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemPrefWritable.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemPrefWritable.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemPrefWritable.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/ItemPrefWritable.java Tue Aug 11 12:04:35 2009 @@ -26,14 +26,14 @@ /** A {@link Writable} encapsulating an item and a preference value. */ public final class ItemPrefWritable implements Writable { - private String itemID; + private long itemID; private float prefValue; public ItemPrefWritable() { // do nothing } - public ItemPrefWritable(String itemID, float prefValue) { + public ItemPrefWritable(long itemID, float prefValue) { this.itemID = itemID; this.prefValue = prefValue; } @@ -42,7 +42,7 @@ this(other.getItemID(), other.getPrefValue()); } - public String getItemID() { + public long getItemID() { return itemID; } @@ -52,13 +52,13 @@ @Override public void write(DataOutput out) throws IOException { - out.writeUTF(itemID); + out.writeLong(itemID); out.writeFloat(prefValue); } @Override public void readFields(DataInput in) throws IOException { - itemID = in.readUTF(); + itemID = in.readLong(); prefValue = in.readFloat(); } Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommendedItemsWritable.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommendedItemsWritable.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommendedItemsWritable.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommendedItemsWritable.java Tue Aug 11 12:04:35 2009 @@ -51,7 +51,7 @@ @Override public void write(DataOutput out) throws IOException { for (RecommendedItem item : recommended) { - out.writeUTF(item.getItemID().toString()); + out.writeLong(item.getItemID()); out.writeFloat(item.getValue()); } @@ -62,7 +62,7 @@ recommended = new ArrayList(); try { do { - String itemID = in.readUTF(); + long itemID = in.readLong(); float value = in.readFloat(); RecommendedItem recommendedItem = new GenericRecommendedItem(itemID, value); recommended.add(recommendedItem); @@ -89,7 +89,7 @@ } else { result.append(','); } - result.append(item.getItemID().toString()); + result.append(item.getItemID()); result.append(':'); result.append(item.getValue()); } Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderJob.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderJob.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderJob.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderJob.java Tue Aug 11 12:04:35 2009 @@ -20,7 +20,7 @@ import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; -import org.apache.hadoop.io.Text; +import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.mapreduce.InputFormat; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; @@ -90,11 +90,11 @@ jobConf.set("mapred.input.dir", StringUtils.escapeString(userIDFilePath.toString())); jobConf.setClass("mapred.mapper.class", RecommenderMapper.class, Mapper.class); - jobConf.setClass("mapred.mapoutput.key.class", Text.class, Object.class); + jobConf.setClass("mapred.mapoutput.key.class", LongWritable.class, Object.class); jobConf.setClass("mapred.mapoutput.value.class", RecommendedItemsWritable.class, Object.class); jobConf.setClass("mapred.reducer.class", IdentityReducer.class, Reducer.class); - jobConf.setClass("mapred.output.key.class", Text.class, Object.class); + jobConf.setClass("mapred.output.key.class", LongWritable.class, Object.class); jobConf.setClass("mapred.output.value.class", RecommendedItemsWritable.class, Object.class); jobConf.setClass("mapred.output.format.class", TextOutputFormat.class, OutputFormat.class); Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderMapper.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderMapper.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderMapper.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/RecommenderMapper.java Tue Aug 11 12:04:35 2009 @@ -21,7 +21,6 @@ import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.LongWritable; -import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; import org.apache.mahout.cf.taste.common.TasteException; import org.apache.mahout.cf.taste.impl.model.file.FileDataModel; @@ -40,14 +39,14 @@ * ID, computes recommendations with the configured {@link Recommender}. The results are output as {@link * RecommendedItemsWritable}.

* - *

Note that there is no corresponding {@link org.apache.hadoop.mapred.Reducer}; this implementation can only + *

Note that there is no corresponding {@link org.apache.hadoop.mapreduce.Reducer}; this implementation can only * partially take advantage of the mapreduce paradigm and only really leverages it for easy parallelization. Therefore, - * use the {@link org.apache.hadoop.mapred.lib.IdentityReducer} when running this on Hadoop.

+ * use the {@link IdentityReducer} when running this on Hadoop.

* * @see RecommenderJob */ public final class RecommenderMapper - extends Mapper { + extends Mapper { static final String RECOMMENDER_CLASS_NAME = "recommenderClassName"; static final String RECOMMENDATIONS_PER_USER = "recommendationsPerUser"; @@ -57,9 +56,9 @@ private int recommendationsPerUser; @Override - protected void map(LongWritable key, Text value, + protected void map(LongWritable key, LongWritable value, Context context) throws IOException, InterruptedException { - String userID = value.toString(); + long userID = value.get(); List recommendedItems; try { recommendedItems = recommender.recommend(userID, recommendationsPerUser); @@ -67,7 +66,7 @@ throw new RuntimeException(te); } RecommendedItemsWritable writable = new RecommendedItemsWritable(recommendedItems); - context.write(new Text(userID), writable); + context.write(value, writable); context.getCounter(ReducerMetrics.USERS_PROCESSED).increment(1L); context.getCounter(ReducerMetrics.RECOMMENDATIONS_MADE).increment(recommendedItems.size()); } Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsMapper.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsMapper.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsMapper.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsMapper.java Tue Aug 11 12:04:35 2009 @@ -24,17 +24,17 @@ import java.io.IOException; public final class SlopeOnePrefsToDiffsMapper - extends Mapper { + extends Mapper { @Override protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { String line = value.toString(); String[] tokens = line.split(","); - String userID = tokens[0]; - String itemID = tokens[1]; + long userID = Long.parseLong(tokens[0]); + long itemID = Long.parseLong(tokens[1]); float prefValue = Float.parseFloat(tokens[2]); - context.write(new Text(userID), new ItemPrefWritable(itemID, prefValue)); + context.write(new LongWritable(userID), new ItemPrefWritable(itemID, prefValue)); } } \ No newline at end of file Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsReducer.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsReducer.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsReducer.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/SlopeOnePrefsToDiffsReducer.java Tue Aug 11 12:04:35 2009 @@ -40,11 +40,11 @@ int size = prefs.size(); for (int i = 0; i < size; i++) { ItemPrefWritable first = prefs.get(i); - String itemAID = first.getItemID(); + long itemAID = first.getItemID(); float itemAValue = first.getPrefValue(); for (int j = i + 1; j < size; j++) { ItemPrefWritable second = prefs.get(j); - String itemBID = second.getItemID(); + long itemBID = second.getItemID(); float itemBValue = second.getPrefValue(); context.write(new ItemItemWritable(itemAID, itemBID), new FloatWritable(itemBValue - itemAValue)); } Copied: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastByIDMap.java (from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java) URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastByIDMap.java?p2=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastByIDMap.java&p1=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java&r1=800705&r2=803081&rev=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastByIDMap.java Tue Aug 11 12:04:35 2009 @@ -18,7 +18,6 @@ package org.apache.mahout.cf.taste.impl.common; import java.io.Serializable; -import java.util.AbstractCollection; import java.util.AbstractSet; import java.util.Arrays; import java.util.Collection; @@ -28,26 +27,19 @@ import java.util.Set; /** - *

This is an optimized {@link Map} implementation, based on algorithms described in Knuth's "Art of Computer - * Programming", Vol. 3, p. 529.

- * - *

It should be faster than {@link java.util.HashMap} in some cases, but not all. Its main feature is a "max size" - * and the ability to transparently, efficiently and semi-intelligently evict old entries when max size is - * exceeded.

- * - *

This class is not a bit thread-safe.

- * - *

This implementation does not allow null as a key or value.

+ * @see FastMap + * @see FastIDSet */ -public final class FastMap implements Map, Serializable, Cloneable { +public final class FastByIDMap implements Serializable, Cloneable { public static final int NO_MAX_SIZE = Integer.MAX_VALUE; private static final double ALLOWED_LOAD_FACTOR = 1.5; /** Dummy object used to represent a key that has been removed. */ - private static final Object REMOVED = new Object(); + private static final long REMOVED = Long.MAX_VALUE; + private static final long NULL = Long.MIN_VALUE; - private K[] keys; + private long[] keys; private V[] values; private int numEntries; private int numSlotsUsed; @@ -55,22 +47,17 @@ private BitSet recentlyAccessed; private final boolean countingAccesses; - /** Creates a new {@link FastMap} with default capacity. */ - public FastMap() { - this(5, NO_MAX_SIZE); + /** Creates a new {@link FastByIDMap} with default capacity. */ + public FastByIDMap() { + this(2, NO_MAX_SIZE); } - public FastMap(int size) { + public FastByIDMap(int size) { this(size, NO_MAX_SIZE); } - public FastMap(Map other) { - this(other.size()); - putAll(other); - } - /** - * Creates a new {@link FastMap} whose capacity can accommodate the given number of entries without rehash.

+ * Creates a new {@link FastByIDMap} whose capacity can accommodate the given number of entries without rehash.

* * @param size desired capacity * @param maxSize max capacity @@ -78,7 +65,7 @@ * or at least half of {@link RandomUtils#MAX_INT_SMALLER_TWIN_PRIME} */ @SuppressWarnings("unchecked") - public FastMap(int size, int maxSize) { + public FastByIDMap(int size, int maxSize) { if (size < 0) { throw new IllegalArgumentException("size must be at least 0"); } @@ -90,37 +77,22 @@ throw new IllegalArgumentException("maxSize must be at least 1"); } int hashSize = RandomUtils.nextTwinPrime((int) (ALLOWED_LOAD_FACTOR * size)); - keys = (K[]) new Object[hashSize]; + keys = new long[hashSize]; + Arrays.fill(keys, NULL); values = (V[]) new Object[hashSize]; this.maxSize = maxSize; this.countingAccesses = maxSize != Integer.MAX_VALUE; this.recentlyAccessed = countingAccesses ? new BitSet(hashSize) : null; } - /** - * This is for the benefit of inner classes. Without it the compiler would just generate a similar synthetic accessor. - * Might as well make it explicit. - */ - K[] getKeys() { - return keys; - } - - /** - * This is for the benefit of inner classes. Without it the compiler would just generate a similar synthetic accessor. - * Might as well make it explicit. - */ - V[] getValues() { - return values; - } - - private int find(Object key) { - int theHashCode = key.hashCode() & 0x7FFFFFFF; // make sure it's positive - K[] keys = this.keys; + private int find(long key) { + int theHashCode = (int) key & 0x7FFFFFFF; // make sure it's positive + long[] keys = this.keys; int hashSize = keys.length; int jump = 1 + theHashCode % (hashSize - 2); int index = theHashCode % hashSize; - K currentKey = keys[index]; - while (currentKey != null && (currentKey == REMOVED || !key.equals(currentKey))) { + long currentKey = keys[index]; + while (currentKey != NULL && (currentKey == REMOVED || key != currentKey)) { if (index < jump) { index += hashSize - jump; } else { @@ -131,9 +103,8 @@ return index; } - @Override - public V get(Object key) { - if (key == null) { + public V get(long key) { + if (key == NULL) { return null; } int index = find(key); @@ -143,22 +114,18 @@ return values[index]; } - @Override public int size() { return numEntries; } - @Override public boolean isEmpty() { return numEntries == 0; } - @Override - public boolean containsKey(Object key) { - return key != null && keys[find(key)] != null; + public boolean containsKey(long key) { + return key != NULL && key != REMOVED && keys[find(key)] != NULL; } - @Override public boolean containsValue(Object value) { if (value == null) { return false; @@ -171,10 +138,11 @@ return false; } - /** @throws NullPointerException if key or value is null */ - @Override - public V put(K key, V value) { - if (key == null || value == null) { + public V put(long key, V value) { + if (key == NULL || key == REMOVED) { + throw new IllegalArgumentException(); + } + if (value == null) { throw new NullPointerException(); } // If less than half the slots are open, let's clear it up @@ -189,7 +157,7 @@ } // Here we may later consider implementing Brent's variation described on page 532 int index = find(key); - if (keys[index] == null) { + if (keys[index] == NULL) { // If size is limited, if (countingAccesses && numEntries >= maxSize) { // and we're too large, clear some old-ish entry @@ -209,7 +177,7 @@ private void clearStaleEntry(int index) { while (true) { - K currentKey; + long currentKey; do { if (index == 0) { index = keys.length - 1; @@ -217,7 +185,7 @@ index--; } currentKey = keys[index]; - } while (currentKey == null || currentKey == REMOVED); + } while (currentKey == NULL || currentKey == REMOVED); if (recentlyAccessed.get(index)) { recentlyAccessed.clear(index); } else { @@ -225,28 +193,20 @@ } } // Delete the entry - ((Object[]) keys)[index] = REMOVED; + keys[index] = REMOVED; numEntries--; values[index] = null; } - @Override - public void putAll(Map map) { - for (Entry entry : map.entrySet()) { - put(entry.getKey(), entry.getValue()); - } - } - - @Override - public V remove(Object key) { - if (key == null) { + public V remove(long key) { + if (key == NULL || key == REMOVED) { return null; } int index = find(key); - if (keys[index] == null) { + if (keys[index] == NULL) { return null; } else { - ((Object[]) keys)[index] = REMOVED; + keys[index] = REMOVED; numEntries--; V oldValue = values[index]; values[index] = null; @@ -256,29 +216,21 @@ // Could un-set recentlyAccessed's bit but doesn't matter } - @Override public void clear() { numEntries = 0; numSlotsUsed = 0; - Arrays.fill(keys, null); + Arrays.fill(keys, NULL); Arrays.fill(values, null); if (countingAccesses) { recentlyAccessed.clear(); } } - @Override - public Set keySet() { - return new KeySet(); + public LongPrimitiveIterator keySetIterator() { + return new KeyIterator(); } - @Override - public Collection values() { - return new ValueCollection(); - } - - @Override - public Set> entrySet() { + public Set> entrySet() { return new EntrySet(); } @@ -293,21 +245,21 @@ rehash(RandomUtils.nextTwinPrime((int) (ALLOWED_LOAD_FACTOR * keys.length))); } - @SuppressWarnings("unchecked") private void rehash(int newHashSize) { - K[] oldKeys = keys; + long[] oldKeys = keys; V[] oldValues = values; numEntries = 0; numSlotsUsed = 0; if (countingAccesses) { recentlyAccessed = new BitSet(newHashSize); } - keys = (K[]) new Object[newHashSize]; + keys = new long[newHashSize]; + Arrays.fill(keys, NULL); values = (V[]) new Object[newHashSize]; int length = oldKeys.length; for (int i = 0; i < length; i++) { - K key = oldKeys[i]; - if (key != null && key != REMOVED) { + long key = oldKeys[i]; + if (key != NULL && key != REMOVED) { put(key, oldValues[i]); } } @@ -321,172 +273,96 @@ throw new IllegalStateException(); } values[lastNext] = null; - ((Object[]) keys)[lastNext] = REMOVED; + keys[lastNext] = REMOVED; numEntries--; } - @SuppressWarnings("unchecked") @Override - public FastMap clone() { - FastMap clone; + public FastByIDMap clone() { + FastByIDMap clone; try { - clone = (FastMap) super.clone(); + clone = (FastByIDMap) super.clone(); } catch (CloneNotSupportedException cnse) { throw new AssertionError(); } - int length = keys.length; - clone.keys = (K[]) new Object[length]; - clone.values = (V[]) new Object[length]; - System.arraycopy(keys, 0, clone.keys, 0, length); - System.arraycopy(values, 0, clone.values, 0, length); - clone.recentlyAccessed = countingAccesses ? new BitSet(length) : null; + clone.keys = keys.clone(); + clone.values = values.clone(); + clone.recentlyAccessed = countingAccesses ? new BitSet(keys.length) : null; return clone; } - private final class EntrySet extends AbstractSet> { - - @Override - public int size() { - return FastMap.this.size(); - } - - @Override - public boolean isEmpty() { - return FastMap.this.isEmpty(); - } - - @Override - public boolean contains(Object o) { - return containsKey(o); - } + private final class KeyIterator implements LongPrimitiveIterator { - @Override - public Iterator> iterator() { - return new EntryIterator(); - } + private int position; + private int lastNext = -1; @Override - public boolean add(Entry t) { - throw new UnsupportedOperationException(); + public boolean hasNext() { + goToNext(); + return position < keys.length; } @Override - public boolean remove(Object o) { - throw new UnsupportedOperationException(); + public Long next() { + return nextLong(); } @Override - public boolean addAll(Collection> ts) { - throw new UnsupportedOperationException(); - } - - @Override - public boolean retainAll(Collection objects) { - throw new UnsupportedOperationException(); - } - - @Override - public boolean removeAll(Collection objects) { - throw new UnsupportedOperationException(); + public long nextLong() { + goToNext(); + lastNext = position; + if (position >= keys.length) { + throw new NoSuchElementException(); + } + return keys[position++]; } @Override - public void clear() { - FastMap.this.clear(); - } - - private final class MapEntry implements Entry { - - private final int index; - - private MapEntry(int index) { - this.index = index; - } - - @Override - public K getKey() { - return getKeys()[index]; - } - - @Override - public V getValue() { - return getValues()[index]; - } - - @Override - public V setValue(V value) { - if (value == null) { - throw new IllegalArgumentException(); - } - V[] values = getValues(); - V oldValue = values[index]; - getValues()[index] = value; - return oldValue; + public long peek() { + goToNext(); + if (position >= keys.length) { + throw new NoSuchElementException(); } + return keys[position]; } - private final class EntryIterator implements Iterator> { - - private int position; - private int lastNext = -1; - - @Override - public boolean hasNext() { - goToNext(); - return position < getKeys().length; - } - - @Override - public Entry next() { - goToNext(); - lastNext = position; - K[] keys = getKeys(); - if (position >= keys.length) { - throw new NoSuchElementException(); - } - return new MapEntry(position++); - } - - private void goToNext() { - V[] values = getValues(); - int length = values.length; - while (position < length && values[position] == null) { - position++; - } - } - - @Override - public void remove() { - iteratorRemove(lastNext); + private void goToNext() { + int length = values.length; + while (position < length && values[position] == null) { + position++; } } + @Override + public void remove() { + iteratorRemove(lastNext); + } } - private final class KeySet extends AbstractSet { + private final class EntrySet extends AbstractSet> { @Override public int size() { - return FastMap.this.size(); + return FastByIDMap.this.size(); } @Override public boolean isEmpty() { - return FastMap.this.isEmpty(); + return FastByIDMap.this.isEmpty(); } @Override public boolean contains(Object o) { - return containsKey(o); + return containsKey((Long) o); } @Override - public Iterator iterator() { - return new KeyIterator(); + public Iterator> iterator() { + return new EntryIterator(); } @Override - public boolean add(K t) { + public boolean add(Map.Entry t) { throw new UnsupportedOperationException(); } @@ -496,7 +372,7 @@ } @Override - public boolean addAll(Collection ts) { + public boolean addAll(Collection> ts) { throw new UnsupportedOperationException(); } @@ -512,100 +388,39 @@ @Override public void clear() { - FastMap.this.clear(); + FastByIDMap.this.clear(); } - private final class KeyIterator implements Iterator { + private final class MapEntry implements Map.Entry { - private int position; - private int lastNext = -1; + private final int index; - @Override - public boolean hasNext() { - goToNext(); - return position < getKeys().length; + private MapEntry(int index) { + this.index = index; } @Override - public K next() { - goToNext(); - lastNext = position; - K[] keys = getKeys(); - if (position >= keys.length) { - throw new NoSuchElementException(); - } - return keys[position++]; + public Long getKey() { + return keys[index]; } - private void goToNext() { - V[] values = getValues(); - int length = values.length; - while (position < length && values[position] == null) { - position++; - } + @Override + public V getValue() { + return values[index]; } @Override - public void remove() { - iteratorRemove(lastNext); + public V setValue(V value) { + if (value == null) { + throw new IllegalArgumentException(); + } + V oldValue = values[index]; + values[index] = value; + return oldValue; } } - } - - private final class ValueCollection extends AbstractCollection { - - @Override - public int size() { - return FastMap.this.size(); - } - - @Override - public boolean isEmpty() { - return FastMap.this.isEmpty(); - } - - @Override - public boolean contains(Object o) { - return containsValue(o); - } - - @Override - public Iterator iterator() { - return new ValueIterator(); - } - - @Override - public boolean add(V v) { - throw new UnsupportedOperationException(); - } - - @Override - public boolean remove(Object o) { - throw new UnsupportedOperationException(); - } - - @Override - public boolean addAll(Collection vs) { - throw new UnsupportedOperationException(); - } - - @Override - public boolean removeAll(Collection objects) { - throw new UnsupportedOperationException(); - } - - @Override - public boolean retainAll(Collection objects) { - throw new UnsupportedOperationException(); - } - - @Override - public void clear() { - FastMap.this.clear(); - } - - private final class ValueIterator implements Iterator { + private final class EntryIterator implements Iterator> { private int position; private int lastNext = -1; @@ -613,22 +428,20 @@ @Override public boolean hasNext() { goToNext(); - return position < getValues().length; + return position < keys.length; } @Override - public V next() { + public Map.Entry next() { goToNext(); lastNext = position; - V[] values = getValues(); - if (position >= values.length) { + if (position >= keys.length) { throw new NoSuchElementException(); } - return values[position++]; + return new MapEntry(position++); } private void goToNext() { - V[] values = getValues(); int length = values.length; while (position < length && values[position] == null) { position++; @@ -639,8 +452,8 @@ public void remove() { iteratorRemove(lastNext); } - } } -} + +} \ No newline at end of file Copied: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastIDSet.java (from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastSet.java) URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastIDSet.java?p2=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastIDSet.java&p1=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastSet.java&r1=800705&r2=803081&rev=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastSet.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastIDSet.java Tue Aug 11 12:04:35 2009 @@ -18,50 +18,31 @@ package org.apache.mahout.cf.taste.impl.common; import java.io.Serializable; -import java.lang.reflect.Array; import java.util.Arrays; -import java.util.Collection; import java.util.Iterator; import java.util.NoSuchElementException; -import java.util.Set; /** - *

This is an optimized {@link Set} implementation, based on algorithms described in Knuth's "Art of Computer - * Programming", Vol. 3, p. 529.

- * - *

It should be faster than {@link java.util.HashSet} in some cases, but not all. It should definitely be more memory - * efficient since that implementation is actually just a {@link java.util.HashMap} underneath mapping values to a dummy - * object.

- * - *

This class is not a bit thread-safe.

- * - *

This implementation does not allow null as a key.

- * - * @see FastMap + * @see FastByIDMap */ -public final class FastSet implements Set, Serializable, Cloneable { +public final class FastIDSet implements Serializable, Cloneable { private static final double ALLOWED_LOAD_FACTOR = 1.5; /** Dummy object used to represent a key that has been removed. */ - private static final Object REMOVED = new Object(); + private static final long REMOVED = Long.MAX_VALUE; + private static final long NULL = Long.MIN_VALUE; - private K[] keys; + private long[] keys; private int numEntries; private int numSlotsUsed; - /** Creates a new {@link FastSet} with default capacity. */ - public FastSet() { - this(5); + /** Creates a new {@link FastIDSet} with default capacity. */ + public FastIDSet() { + this(2); } - public FastSet(Collection c) { - this(c.size()); - addAll(c); - } - - @SuppressWarnings("unchecked") - public FastSet(int size) { + public FastIDSet(int size) { if (size < 0) { throw new IllegalArgumentException("size must be at least 0"); } @@ -70,25 +51,18 @@ throw new IllegalArgumentException("size must be less than " + max); } int hashSize = RandomUtils.nextTwinPrime((int) (ALLOWED_LOAD_FACTOR * size)); - keys = (K[]) new Object[hashSize]; - } - - /** - * This is for the benefit of inner classes. Without it the compiler would just generate a similar synthetic accessor. - * Might as well make it explicit. - */ - K[] getKeys() { - return keys; + keys = new long[hashSize]; + Arrays.fill(keys, NULL); } - private int find(Object key) { - int theHashCode = key.hashCode() & 0x7FFFFFFF; // make sure it's positive - K[] keys = this.keys; + private int find(long key) { + int theHashCode = (int) key & 0x7FFFFFFF; // make sure it's positive + long[] keys = this.keys; int hashSize = keys.length; int jump = 1 + theHashCode % (hashSize - 2); int index = theHashCode % hashSize; - K currentKey = keys[index]; - while (currentKey != null && (currentKey == REMOVED || !key.equals(currentKey))) { + long currentKey = keys[index]; + while (currentKey != NULL && (currentKey == REMOVED || key != currentKey)) { if (index < jump) { index += hashSize - jump; } else { @@ -99,26 +73,21 @@ return index; } - @Override public int size() { return numEntries; } - @Override public boolean isEmpty() { return numEntries == 0; } - @Override - public boolean contains(Object key) { - return key != null && keys[find(key)] != null; + public boolean contains(long key) { + return key != NULL && key != REMOVED && keys[find(key)] != NULL; } - /** @throws NullPointerException if key is null */ - @Override - public boolean add(K key) { - if (key == null) { - throw new NullPointerException(); + public boolean add(long key) { + if (key == NULL || key == REMOVED) { + throw new IllegalArgumentException(); } // If less than half the slots are open, let's clear it up if (numSlotsUsed * ALLOWED_LOAD_FACTOR >= keys.length) { @@ -132,7 +101,7 @@ } // Here we may later consider implementing Brent's variation described on page 532 int index = find(key); - if (keys[index] == null) { + if (keys[index] == NULL) { keys[index] = key; numEntries++; numSlotsUsed++; @@ -141,100 +110,96 @@ return false; } - @Override - public Iterator iterator() { + public LongPrimitiveIterator iterator() { return new KeyIterator(); } - @Override - public boolean remove(Object key) { - if (key == null) { + public long[] toArray() { + long[] result = new long[numEntries]; + for (int i = 0, position = 0; i < result.length; i++) { + while (keys[position] == NULL || keys[position] == REMOVED) { + position++; + } + result[i] = keys[position++]; + } + return result; + } + + public boolean remove(long key) { + if (key == NULL || key == REMOVED) { return false; } int index = find(key); - if (keys[index] == null) { + if (keys[index] == NULL) { return false; } else { - ((Object[]) keys)[index] = REMOVED; + keys[index] = REMOVED; numEntries--; return true; } - // Could un-set recentlyAccessed's bit but doesn't matter } - @Override - public boolean containsAll(Collection c) { - for (Object o : c) { - if (o == null || keys[find(o)] == null) { - return false; + public boolean addAll(long[] c) { + boolean changed = false; + for (long k : c) { + if (add(k)) { + changed = true; } } - return true; + return changed; } - @Override - public boolean addAll(Collection c) { + public boolean addAll(FastIDSet c) { boolean changed = false; - for (K k : c) { - if (add(k)) { - changed = true; + for (long k : c.keys) { + if (k != NULL && k != REMOVED) { + if (add(k)) { + changed = true; + } } } return changed; } - @Override - public boolean retainAll(Collection c) { + public boolean removeAll(long[] c) { boolean changed = false; - Iterator iterator = iterator(); - while (iterator.hasNext()) { - K k = iterator.next(); - if (!c.contains(k)) { - iterator.remove(); + for (long o : c) { + if (remove(o)) { changed = true; } } return changed; } - @Override - public boolean removeAll(Collection c) { + public boolean removeAll(FastIDSet c) { boolean changed = false; - for (Object o : c) { - if (remove(o)) { + for (long k : c.keys) { + if (k != NULL && k != REMOVED) { + if (remove(k)) { + changed = true; + } + } + } + return changed; + } + + public boolean retainAll(FastIDSet c) { + boolean changed = false; + for (int i = 0; i < keys.length; i++) { + long k = keys[i]; + if (k != NULL && k != REMOVED && !c.contains(k)) { + keys[i] = REMOVED; + numEntries--; changed = true; } } return changed; } - @Override public void clear() { numEntries = 0; numSlotsUsed = 0; - Arrays.fill(keys, null); - } - - @Override - public Object[] toArray() { - return toArray(new Object[numEntries]); - } - - @Override - @SuppressWarnings("unchecked") - public T[] toArray(T[] a) { - if (a.length < numEntries) { - a = (T[]) Array.newInstance(a.getClass().getComponentType(), numEntries); - } - int keyOffset = 0; - int resultOffset = 0; - while (resultOffset < a.length) { - K key = keys[keyOffset++]; - if (key != null && key != REMOVED) { - a[resultOffset++] = (T) key; - } - } - return a; + Arrays.fill(keys, NULL); } private void growAndRehash() { @@ -248,48 +213,50 @@ rehash(RandomUtils.nextTwinPrime((int) (ALLOWED_LOAD_FACTOR * numEntries))); } - @SuppressWarnings("unchecked") private void rehash(int newHashSize) { - K[] oldKeys = keys; + long[] oldKeys = keys; numEntries = 0; numSlotsUsed = 0; - keys = (K[]) new Object[newHashSize]; + keys = new long[newHashSize]; + Arrays.fill(keys, NULL); int length = oldKeys.length; for (int i = 0; i < length; i++) { - K key = oldKeys[i]; - if (key != null && key != REMOVED) { + long key = oldKeys[i]; + if (key != NULL && key != REMOVED) { add(key); } } } - /** Convenience method to quickly compute just the size of the intersection with another {@link FastSet}. */ - @SuppressWarnings("unchecked") - public int intersectionSize(FastSet other) { + /** + * Convenience method to quickly compute just the size of the intersection with another {@link FastIDSet}. + * + * @param other {@link FastIDSet} to intersect with + * @return number of elements in intersection + */ + public int intersectionSize(FastIDSet other) { int count = 0; - for (K key : (K[]) other.keys) { - if (key != null && key != REMOVED && keys[find(key)] != null) { + for (long key : other.keys) { + if (key != NULL && key != REMOVED && keys[find(key)] != NULL) { count++; } } return count; } - @SuppressWarnings("unchecked") @Override - public FastSet clone() { - FastSet clone; + public FastIDSet clone() { + FastIDSet clone; try { - clone = (FastSet) super.clone(); + clone = (FastIDSet) super.clone(); } catch (CloneNotSupportedException cnse) { throw new AssertionError(); } - clone.keys = (K[]) new Object[keys.length]; - System.arraycopy(keys, 0, clone.keys, 0, keys.length); + clone.keys = keys.clone(); return clone; } - private final class KeyIterator implements Iterator { + private final class KeyIterator implements LongPrimitiveIterator { private int position; private int lastNext = -1; @@ -297,24 +264,36 @@ @Override public boolean hasNext() { goToNext(); - return position < getKeys().length; + return position < keys.length; + } + + @Override + public Long next() { + return nextLong(); } @Override - public K next() { + public long nextLong() { goToNext(); lastNext = position; - K[] keys = getKeys(); if (position >= keys.length) { throw new NoSuchElementException(); } return keys[position++]; } + @Override + public long peek() { + goToNext(); + if (position >= keys.length) { + throw new NoSuchElementException(); + } + return keys[position]; + } + private void goToNext() { - K[] keys = getKeys(); int length = keys.length; - while (position < length && (keys[position] == null || keys[position] == REMOVED)) { + while (position < length && (keys[position] == NULL || keys[position] == REMOVED)) { position++; } } @@ -327,11 +306,14 @@ if (lastNext < 0) { throw new IllegalStateException(); } - ((Object[]) keys)[lastNext] = REMOVED; + keys[lastNext] = REMOVED; numEntries--; } - } + public Iterator iterator() { + return new KeyIterator(); + } + } -} +} \ No newline at end of file Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/FastMap.java Tue Aug 11 12:04:35 2009 @@ -57,7 +57,7 @@ /** Creates a new {@link FastMap} with default capacity. */ public FastMap() { - this(5, NO_MAX_SIZE); + this(2, NO_MAX_SIZE); } public FastMap(int size) { @@ -97,22 +97,6 @@ this.recentlyAccessed = countingAccesses ? new BitSet(hashSize) : null; } - /** - * This is for the benefit of inner classes. Without it the compiler would just generate a similar synthetic accessor. - * Might as well make it explicit. - */ - K[] getKeys() { - return keys; - } - - /** - * This is for the benefit of inner classes. Without it the compiler would just generate a similar synthetic accessor. - * Might as well make it explicit. - */ - V[] getValues() { - return values; - } - private int find(Object key) { int theHashCode = key.hashCode() & 0x7FFFFFFF; // make sure it's positive K[] keys = this.keys; @@ -405,12 +389,12 @@ @Override public K getKey() { - return getKeys()[index]; + return keys[index]; } @Override public V getValue() { - return getValues()[index]; + return values[index]; } @Override @@ -418,9 +402,8 @@ if (value == null) { throw new IllegalArgumentException(); } - V[] values = getValues(); V oldValue = values[index]; - getValues()[index] = value; + values[index] = value; return oldValue; } } @@ -433,14 +416,13 @@ @Override public boolean hasNext() { goToNext(); - return position < getKeys().length; + return position < keys.length; } @Override public Entry next() { goToNext(); lastNext = position; - K[] keys = getKeys(); if (position >= keys.length) { throw new NoSuchElementException(); } @@ -448,7 +430,6 @@ } private void goToNext() { - V[] values = getValues(); int length = values.length; while (position < length && values[position] == null) { position++; @@ -523,14 +504,13 @@ @Override public boolean hasNext() { goToNext(); - return position < getKeys().length; + return position < keys.length; } @Override public K next() { goToNext(); lastNext = position; - K[] keys = getKeys(); if (position >= keys.length) { throw new NoSuchElementException(); } @@ -538,7 +518,6 @@ } private void goToNext() { - V[] values = getValues(); int length = values.length; while (position < length && values[position] == null) { position++; @@ -613,14 +592,13 @@ @Override public boolean hasNext() { goToNext(); - return position < getValues().length; + return position < values.length; } @Override public V next() { goToNext(); lastNext = position; - V[] values = getValues(); if (position >= values.length) { throw new NoSuchElementException(); } @@ -628,7 +606,6 @@ } private void goToNext() { - V[] values = getValues(); int length = values.length; while (position < length && values[position] == null) { position++; Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/IteratorUtils.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/IteratorUtils.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/IteratorUtils.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/IteratorUtils.java Tue Aug 11 12:04:35 2009 @@ -37,6 +37,25 @@ return iterableToList(iterable, null); } + public static long[] longIteratorToList(LongPrimitiveIterator iterator) { + long[] result = new long[5]; + int size = 0; + while (iterator.hasNext()) { + if (size == result.length) { + long[] newResult = new long[result.length * 2]; + System.arraycopy(result, 0, newResult, 0, result.length); + result = newResult; + } + result[size++] = iterator.next(); + } + if (size != result.length) { + long[] newResult = new long[size]; + System.arraycopy(result, 0, newResult, 0, size); + result = newResult; + } + return result; + } + /** * @param iterable {@link Iterable} whose contents are to be put into a {@link List} * @param comparator {@link Comparator} defining the sort order of the returned {@link List} Copied: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPair.java (from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java) URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPair.java?p2=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPair.java&p1=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java&r1=800705&r2=803081&rev=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPair.java Tue Aug 11 12:04:35 2009 @@ -17,49 +17,42 @@ package org.apache.mahout.cf.taste.impl.common; -/** A simple (ordered) pair of two objects. Elements may be null. */ -public final class Pair { +import java.io.Serializable; - private final A first; - private final B second; +/** A simple (ordered) pair of longs. */ +public final class LongPair implements Comparable, Serializable { - public Pair(A first, B second) { + private final long first; + private final long second; + + public LongPair(long first, long second) { this.first = first; this.second = second; } - public A getFirst() { + public long getFirst() { return first; } - public B getSecond() { + public long getSecond() { return second; } @Override public boolean equals(Object obj) { - if (!(obj instanceof Pair)) { + if (!(obj instanceof LongPair)) { return false; } - Pair otherPair = (Pair) obj; - return isEqualOrNulls(first, otherPair.first) && - isEqualOrNulls(second, otherPair.second); - } - - private static boolean isEqualOrNulls(Object obj1, Object obj2) { - return obj1 == null ? obj2 == null : obj1.equals(obj2); + LongPair otherPair = (LongPair) obj; + return first == otherPair.first && second == otherPair.second; } @Override public int hashCode() { - int firstHash = hashCodeNull(first); + int firstHash = RandomUtils.hashLong(first); // Flip top and bottom 16 bits; this makes the hash function probably different // for (a,b) versus (b,a) - return (firstHash >>> 16 | firstHash << 16) ^ hashCodeNull(second); - } - - private static int hashCodeNull(Object obj) { - return obj == null ? 0 : obj.hashCode(); + return (firstHash >>> 16 | firstHash << 16) ^ RandomUtils.hashLong(second); } @Override @@ -67,4 +60,15 @@ return '(' + String.valueOf(first) + ',' + second + ')'; } -} + @Override + public int compareTo(LongPair o) { + if (first < o.first) { + return -1; + } else if (first > o.first) { + return 1; + } else { + return second < o.second ? -1 : second > o.second ? 1 : 0; + } + } + +} \ No newline at end of file Copied: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveArrayIterator.java (from r800705, lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/ArrayIterator.java) URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveArrayIterator.java?p2=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveArrayIterator.java&p1=lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/ArrayIterator.java&r1=800705&r2=803081&rev=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/ArrayIterator.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveArrayIterator.java Tue Aug 11 12:04:35 2009 @@ -17,23 +17,24 @@ package org.apache.mahout.cf.taste.impl.common; -import java.util.Arrays; -import java.util.Iterator; import java.util.NoSuchElementException; -/**

Simple, fast {@link Iterator} for an array.

*/ -public final class ArrayIterator implements SkippingIterator, Iterable { +/** + * While long[] is an Iterable, it is not an Iterable<Long>. This adapter class + * addresses that. + */ +public final class LongPrimitiveArrayIterator implements LongPrimitiveIterator, SkippingIterator { - private final T[] array; + private final long[] array; private int position; private final int max; /** - *

Creates an {@link ArrayIterator} over an entire array.

+ *

Creates an {@link LongPrimitiveArrayIterator} over an entire array.

* * @param array array to iterate over */ - public ArrayIterator(T[] array) { + public LongPrimitiveArrayIterator(long[] array) { if (array == null) { throw new IllegalArgumentException("array is null"); } @@ -48,13 +49,26 @@ } @Override - public T next() { + public Long next() { + return nextLong(); + } + + @Override + public long nextLong() { if (position >= array.length) { throw new NoSuchElementException(); } return array[position++]; } + @Override + public long peek() { + if (position >= array.length) { + throw new NoSuchElementException(); + } + return array[position]; + } + /** * @throws UnsupportedOperationException */ @@ -71,13 +85,8 @@ } @Override - public Iterator iterator() { - return this; - } - - @Override public String toString() { - return "ArrayIterator[" + Arrays.toString(array) + '@' + position + ']'; + return "LongPrimitiveArrayIterator"; } -} +} \ No newline at end of file Added: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveIterator.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveIterator.java?rev=803081&view=auto ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveIterator.java (added) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/LongPrimitiveIterator.java Tue Aug 11 12:04:35 2009 @@ -0,0 +1,41 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.mahout.cf.taste.impl.common; + +import java.util.Iterator; + +/** + * Adds notion of iterating over long primitives in the style of an + * {@link Iterator} -- as opposed to iterating over {@link Long}. Implementations of + * this interface however also implement {@link Iterator} and {@link Iterable} over + * {@link Long} for convenience. + */ +public interface LongPrimitiveIterator extends Iterator { + + /** + * @return next long in iteration + * @throws java.util.NoSuchElementException if no more elements exist in the iteration + */ + long nextLong(); + + /** + * @return next long in iteration without advancing iteration + */ + long peek(); + +} \ No newline at end of file Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/Pair.java Tue Aug 11 12:04:35 2009 @@ -17,8 +17,10 @@ package org.apache.mahout.cf.taste.impl.common; +import java.io.Serializable; + /** A simple (ordered) pair of two objects. Elements may be null. */ -public final class Pair { +public final class Pair implements Serializable { private final A first; private final B second; Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RandomUtils.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RandomUtils.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RandomUtils.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RandomUtils.java Tue Aug 11 12:04:35 2009 @@ -55,6 +55,24 @@ return Float.floatToIntBits(value); } + public static int hashLong(long value) { + return (int) (value ^ (value >>> 32)); + } + + /** + * See String.hashCode() from Apache Harmony + */ + public static long hashStringToLong(String value) { + long hash = 0L; + int multiplier = 1; + for (int i = value.length() - 1; i >= 0; i--) { + hash += value.charAt(i) * multiplier; + int shifted = multiplier << 5; + multiplier = shifted - multiplier; + } + return hash; + } + /** *

Finds next-largest "twin primes": numbers p and p+2 such that both are prime. Finds the smallest such p such * that the smaller twin, p, is greater than or equal to n. Returns p+2, the larger of the two twins.

Modified: lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RefreshHelper.java URL: http://svn.apache.org/viewvc/lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RefreshHelper.java?rev=803081&r1=803080&r2=803081&view=diff ============================================================================== --- lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RefreshHelper.java (original) +++ lucene/mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/common/RefreshHelper.java Tue Aug 11 12:04:35 2009 @@ -23,6 +23,7 @@ import java.util.ArrayList; import java.util.Collection; +import java.util.HashSet; import java.util.List; import java.util.concurrent.Callable; import java.util.concurrent.locks.ReentrantLock; @@ -87,13 +88,13 @@ } /** - * Creates a new and empty {@link FastSet} of size 3 if the method parameter is null. + * Creates a new and empty {@link Collection} if the method parameter is null. * * @param currentAlreadyRefreshed {@link Refreshable}s to refresh later on - * @return an empty {@link FastSet} if the method param was null or the unmodified method param. + * @return an empty {@link Collection} if the method param was null or the unmodified method param. */ public static Collection buildRefreshed(Collection currentAlreadyRefreshed) { - return currentAlreadyRefreshed == null ? new FastSet(2) : currentAlreadyRefreshed; + return currentAlreadyRefreshed == null ? new HashSet(3) : currentAlreadyRefreshed; } /** @@ -104,7 +105,6 @@ * @param refreshable the {@link Refreshable} to potentially add and refresh */ public static void maybeRefresh(Collection alreadyRefreshed, Refreshable refreshable) { - log.debug("In maybeRefresh({})", refreshable); if (!alreadyRefreshed.contains(refreshable)) { alreadyRefreshed.add(refreshable); log.info("Added refreshable: {}", refreshable);