Return-Path: X-Original-To: apmail-usergrid-commits-archive@minotaur.apache.org Delivered-To: apmail-usergrid-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C501D17855 for ; Thu, 19 Mar 2015 22:19:55 +0000 (UTC) Received: (qmail 8243 invoked by uid 500); 19 Mar 2015 22:19:53 -0000 Delivered-To: apmail-usergrid-commits-archive@usergrid.apache.org Received: (qmail 8195 invoked by uid 500); 19 Mar 2015 22:19:53 -0000 Mailing-List: contact commits-help@usergrid.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@usergrid.incubator.apache.org Delivered-To: mailing list commits@usergrid.incubator.apache.org Received: (qmail 6955 invoked by uid 99); 19 Mar 2015 22:19:52 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Mar 2015 22:19:52 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id E874AE1945; Thu, 19 Mar 2015 22:19:51 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: grey@apache.org To: commits@usergrid.apache.org Date: Thu, 19 Mar 2015 22:20:40 -0000 Message-Id: In-Reply-To: <5a2dd25d654347dc9aa108f7e8cbc2b3@git.apache.org> References: <5a2dd25d654347dc9aa108f7e8cbc2b3@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [50/50] [abbrv] incubator-usergrid git commit: Merge branch 'USERGRID-405' of https://git-wip-us.apache.org/repos/asf/incubator-usergrid into USERGRID-493 Merge branch 'USERGRID-405' of https://git-wip-us.apache.org/repos/asf/incubator-usergrid into USERGRID-493 # By Todd Nine (38) and Shawn Feldman (18) # Via Shawn Feldman (7) and others * 'USERGRID-405' of https://git-wip-us.apache.org/repos/asf/incubator-usergrid: (56 commits) Optimizes import to avoid merge errors Removed exception swallowing Added test to prove issue Fixes issue with repair logic with incorrect assumptions of version ordering. Versions can appear in any order during arbitrary order by statements. Removed Hystrix. Causing thread OOM issues and not really used in production. Revert "fix for list types changing from string to int or whatever" Second pass at fixing the merge fix for list types changing from string to int or whatever Added REST tier invocation for migrations. WIP overwrite Added more tests Fixes bug in return type Renames listener to complete. Refactored plugins to allow a "bootstrap" system plugin. Fixes bug with delete marking and incorrect return value Refactored packages Fixes incorrect system initialization order Removed Hystrix for performance testing. Upgraded plexus utils and fixes build concurrency issues with surefire Temporarily removed test. Need to rebuild with plugin structure. ... Project: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/commit/3eae925d Tree: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/tree/3eae925d Diff: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/diff/3eae925d Branch: refs/heads/USERGRID-493 Commit: 3eae925da199e04ccab1b4600de71b1c1eede8c9 Parents: ce6626d 8ea46ba Author: GERey Authored: Thu Mar 19 15:13:14 2015 -0700 Committer: GERey Committed: Thu Mar 19 15:13:14 2015 -0700 ---------------------------------------------------------------------- .../main/dist/init_instance/init_rest_server.sh | 6 +- stack/core/pom.xml | 10 + .../AllApplicationsObservable.java | 39 ++ .../usergrid/corepersistence/CoreModule.java | 88 ++- .../corepersistence/CpEntityManagerFactory.java | 109 ++- .../corepersistence/CpManagerCache.java | 55 +- .../corepersistence/CpRelationManager.java | 7 +- .../usergrid/corepersistence/ManagerCache.java | 12 +- .../events/EntityVersionDeletedHandler.java | 16 +- .../migration/CoreDataVersions.java | 43 ++ .../migration/CoreMigration.java | 53 ++ .../migration/CoreMigrationPlugin.java | 64 ++ .../migration/EntityDataMigration.java | 147 ---- .../migration/EntityTypeMappingMigration.java | 78 ++- .../migration/GraphShardVersionMigration.java | 152 ----- .../migration/MigrationModuleVersionPlugin.java | 133 ++++ .../migration/MigrationSystemVersions.java | 43 ++ .../corepersistence/migration/Versions.java | 14 - .../results/FilteringLoader.java | 50 +- .../rx/AllEntitiesInSystemObservable.java | 101 --- .../rx/ApplicationObservable.java | 128 ---- .../rx/EdgesFromSourceObservable.java | 63 -- .../rx/EdgesToTargetObservable.java | 63 -- .../corepersistence/rx/TargetIdObservable.java | 66 -- .../rx/impl/AbstractGraphVisitorImpl.java | 107 +++ .../rx/impl/AllApplicationsObservableImpl.java | 142 ++++ .../rx/impl/AllEntitiesInSystemImpl.java | 62 ++ .../rx/impl/AllNodesInGraphImpl.java | 55 ++ .../corepersistence/util/CpEntityMapUtils.java | 19 +- .../corepersistence/util/CpNamingUtils.java | 2 + .../persistence/EntityManagerFactory.java | 20 - .../cassandra/EntityManagerFactoryImpl.java | 23 - .../corepersistence/StaleIndexCleanupTest.java | 48 ++ .../migration/EntityDataMigrationIT.java | 262 ------- .../migration/EntityTypeMappingMigrationIT.java | 183 ++--- .../migration/GraphShardVersionMigrationIT.java | 226 ------ .../MigrationModuleVersionPluginTest.java | 259 +++++++ .../migration/MigrationTestRule.java | 99 --- .../migration/TestProgressObserver.java | 71 -- .../rx/AllEntitiesInSystemObservableIT.java | 41 +- .../rx/ApplicationObservableTestIT.java | 14 +- .../rx/EdgesFromSourceObservableIT.java | 10 +- .../rx/EdgesToTargetObservableIT.java | 8 +- .../rx/TargetIdObservableTestIT.java | 7 +- .../org/apache/usergrid/persistence/GeoIT.java | 5 +- .../usergrid/persistence/LargeEntityIT.java | 3 +- .../collection/EntityCollectionManager.java | 6 - .../EntityCollectionManagerFactory.java | 27 +- .../collection/EntityCollectionManagerSync.java | 49 -- .../persistence/collection/MvccEntity.java | 5 +- .../persistence/collection/MvccLogEntry.java | 1 + .../cache/CachedEntityCollectionManager.java | 7 - .../collection/event/EntityVersionDeleted.java | 3 +- .../collection/guice/CollectionModule.java | 45 +- .../persistence/collection/guice/Write.java | 17 - .../collection/guice/WriteUpdate.java | 17 - .../EntityCollectionManagerFactoryImpl.java | 71 +- .../impl/EntityCollectionManagerImpl.java | 119 +--- .../impl/EntityCollectionManagerSyncImpl.java | 71 -- .../collection/impl/EntityDeletedTask.java | 8 +- .../impl/EntityVersionCleanupTask.java | 191 +++--- .../mvcc/MvccEntitySerializationStrategy.java | 102 --- .../mvcc/stage/delete/MarkCommit.java | 2 +- .../mvcc/stage/write/WriteCommit.java | 9 +- .../collection/mvcc/stage/write/WriteStart.java | 6 +- .../collection/serialization/EntityRepair.java | 38 -- .../MvccEntitySerializationStrategy.java | 119 ++++ .../serialization/OptimisticUpdate.java | 23 - .../UniqueValueSerializationStrategy.java | 46 +- .../serialization/UniqueValueSet.java | 4 + .../impl/CollectionDataVersions.java | 44 ++ .../serialization/impl/EntityRepairImpl.java | 149 ---- .../impl/EntityVersionSerializer.java | 2 - .../serialization/impl/FieldSerializer.java | 118 ---- .../MvccEntitySerializationStrategyImpl.java | 24 +- ...vccEntitySerializationStrategyProxyImpl.java | 151 ++-- .../MvccEntitySerializationStrategyV1Impl.java | 9 +- .../MvccEntitySerializationStrategyV2Impl.java | 8 +- .../MvccEntitySerializationStrategyV3Impl.java | 615 +++++++++++++++++ .../serialization/impl/SerializationModule.java | 85 ++- .../serialization/impl/UniqueFieldEntry.java | 61 ++ .../impl/UniqueFieldEntrySerializer.java | 141 ++++ .../impl/UniqueFieldRowKeySerializer.java | 122 ++++ .../UniqueValueSerializationStrategyImpl.java | 254 +++++-- .../impl/migration/CollectionMigration.java | 53 ++ .../migration/CollectionMigrationPlugin.java | 68 ++ .../impl/migration/EntityIdScope.java | 49 ++ .../migration/MvccEntityDataMigrationImpl.java | 283 ++++++++ .../collection/util/EntityUtils.java | 19 +- .../collection/EntityCollectionManagerIT.java | 104 +-- .../EntityCollectionManagerSyncIT.java | 193 ------ .../collection/guice/TestCollectionModule.java | 22 +- .../impl/EntityVersionCleanupTaskTest.java | 680 ++++++++----------- .../impl/EntityVersionCreatedTaskTest.java | 2 - .../mvcc/stage/delete/MarkCommitTest.java | 2 +- .../mvcc/stage/write/FieldSerializerTest.java | 52 -- .../write/UniqueFieldRowKeySerializerTest.java | 52 ++ ...niqueValueSerializationStrategyImplTest.java | 149 +++- .../mvcc/stage/write/WriteCommitTest.java | 2 +- .../mvcc/stage/write/WriteStartTest.java | 6 +- .../serialization/EntityRepairImplTest.java | 147 ---- .../impl/LogEntryIteratorTest.java | 3 +- ...MvccEntitySerializationStrategyImplTest.java | 117 +--- ...cEntitySerializationStrategyProxyV1Test.java | 85 --- ...ntitySerializationStrategyProxyV1_3Test.java | 83 +++ ...cEntitySerializationStrategyProxyV2Test.java | 83 --- ...ntitySerializationStrategyProxyV2_3Test.java | 84 +++ ...ccEntitySerializationStrategyV1ImplTest.java | 80 ++- ...ccEntitySerializationStrategyV2ImplTest.java | 75 +- .../MvccEntitySerializationStrategyV2Test.java | 7 +- ...ccEntitySerializationStrategyV3ImplTest.java | 74 ++ .../impl/UniqueFieldEntrySerializerTest.java | 108 +++ ...ctMvccEntityDataMigrationV1ToV3ImplTest.java | 215 ++++++ .../MvccEntityDataMigrationV1ToV3ImplTest.java | 105 +++ .../MvccEntityDataMigrationV2ToV3ImplTest.java | 99 +++ .../collection/util/LogEntryMock.java | 39 +- .../collection/util/UniqueValueEntryMock.java | 161 +++++ .../collection/util/VersionGenerator.java | 55 ++ .../usergrid/persistence/core/CPManager.java | 28 + .../astyanax/DynamicCompositeParserImpl.java | 81 +++ .../core/astyanax/FieldBufferBuilder.java | 15 + .../core/astyanax/FieldBufferParser.java | 13 + .../core/astyanax/MultiRowColumnIterator.java | 12 +- .../persistence/core/guice/CommonModule.java | 17 +- .../persistence/core/guice/CurrentImpl.java | 42 -- .../persistence/core/guice/PreviousImpl.java | 42 -- .../core/hystrix/HystrixCassandra.java | 94 --- .../migration/data/AbstractMigrationPlugin.java | 128 ++++ .../core/migration/data/DataMigration.java | 97 +-- .../migration/data/DataMigrationManager.java | 18 +- .../data/DataMigrationManagerImpl.java | 234 ++++--- .../migration/data/MigrationDataProvider.java | 43 ++ .../core/migration/data/MigrationInfoCache.java | 52 ++ .../migration/data/MigrationInfoCacheImpl.java | 84 +++ .../data/MigrationInfoSerialization.java | 28 +- .../data/MigrationInfoSerializationImpl.java | 67 +- .../core/migration/data/MigrationPlugin.java | 56 ++ .../migration/data/MigrationRelationship.java | 100 +++ .../core/migration/data/PluginPhase.java | 42 ++ .../core/migration/data/ProgressObserver.java | 63 ++ .../core/migration/data/VersionedData.java | 38 ++ .../migration/data/VersionedMigrationSet.java | 153 +++++ .../core/scope/ApplicationScope.java | 1 + .../persistence/core/task/TaskExecutor.java | 4 +- .../core/guice/DataMigrationResetRule.java | 88 +++ .../core/guice/MaxMigrationModule.java | 39 -- .../core/guice/MaxMigrationVersion.java | 40 -- .../core/guice/MigrationManagerRule.java | 7 + .../core/guice/TestCommonModule.java | 1 + .../persistence/core/guice/TestModule.java | 5 +- .../data/DataMigrationManagerImplTest.java | 350 ++++++---- .../data/MigrationInfoSerializationTest.java | 32 +- .../data/TestMigrationDataProvider.java | 61 ++ .../migration/data/TestProgressObserver.java | 89 +++ .../data/VersionedMigrationSetTest.java | 198 ++++++ .../persistence/core/util/IdGenerator.java | 51 ++ .../persistence/graph/GraphManager.java | 3 +- .../persistence/graph/GraphManagerFactory.java | 2 + .../persistence/graph/guice/GraphModule.java | 103 ++- .../graph/impl/GraphManagerImpl.java | 32 +- .../graph/impl/stage/EdgeDeleteRepairImpl.java | 9 +- .../graph/impl/stage/EdgeMetaRepairImpl.java | 16 +- .../impl/stage/NodeDeleteListenerImpl.java | 16 +- .../EdgeMetadataSerialization.java | 3 +- .../graph/serialization/EdgesObservable.java | 33 + .../graph/serialization/TargetIdObservable.java | 38 ++ .../EdgeMetadataSerializationProxyImpl.java | 161 +++-- .../impl/EdgeMetadataSerializationV1Impl.java | 6 + .../impl/EdgeMetadataSerializationV2Impl.java | 6 + .../serialization/impl/EdgesObservableImpl.java | 80 +++ .../serialization/impl/GraphDataVersions.java | 43 ++ .../impl/GraphManagerFactoryImpl.java | 95 +++ .../impl/NodeSerializationImpl.java | 36 +- .../impl/TargetIdObservableImpl.java | 72 ++ .../impl/migration/EdgeDataMigrationImpl.java | 149 ++++ .../impl/migration/GraphMigration.java | 53 ++ .../impl/migration/GraphMigrationPlugin.java | 69 ++ .../serialization/impl/migration/GraphNode.java | 39 ++ .../shard/count/NodeShardApproximationImpl.java | 4 +- .../NodeShardCounterSerializationImpl.java | 25 +- .../shard/impl/NodeShardAllocationImpl.java | 16 +- .../shard/impl/ShardGroupCompactionImpl.java | 64 +- .../persistence/graph/GraphManagerIT.java | 21 +- .../persistence/graph/GraphManagerLoadTest.java | 13 +- .../graph/GraphManagerShardConsistencyIT.java | 10 +- .../graph/GraphManagerShardingIT.java | 13 +- .../graph/GraphManagerStressTest.java | 11 +- .../graph/guice/TestGraphModule.java | 27 +- .../graph/impl/EdgeDeleteListenerTest.java | 11 +- .../graph/impl/NodeDeleteListenerTest.java | 9 +- .../graph/impl/stage/EdgeDeleteRepairTest.java | 7 +- .../graph/impl/stage/EdgeMetaRepairTest.java | 29 +- .../EdgeMetaDataSerializationProxyV1Test.java | 14 +- .../EdgeMetaDataSerializationProxyV2Test.java | 21 +- .../EdgeMetaDataSerializationV1Test.java | 7 +- .../EdgeMetaDataSerializationV2Test.java | 7 +- .../EdgeMetadataSerializationTest.java | 39 +- .../EdgeSerializationChopTest.java | 7 +- .../serialization/EdgeSerializationTest.java | 21 +- .../serialization/NodeSerializationTest.java | 15 +- .../migration/EdgeDataMigrationImplTest.java | 177 +++++ .../impl/shard/EdgeShardSerializationTest.java | 7 +- .../impl/shard/NodeShardAllocationTest.java | 27 +- .../impl/shard/NodeShardCacheTest.java | 7 +- .../impl/shard/ShardGroupCompactionTest.java | 7 +- .../shard/count/NodeShardApproximationTest.java | 9 +- .../NodeShardCounterSerializationTest.java | 6 +- .../shard/impl/ShardEntryGroupIteratorTest.java | 15 +- ...rceDirectedEdgeDescendingComparatorTest.java | 23 +- ...getDirectedEdgeDescendingComparatorTest.java | 23 +- .../graph/test/util/EdgeTestUtils.java | 29 +- .../usergrid/persistence/map/MapManager.java | 25 +- .../persistence/map/guice/MapModule.java | 8 +- .../map/impl/MapManagerFactoryImpl.java | 29 +- .../persistence/map/guice/TestMapModule.java | 3 +- .../persistence/model/entity/Entity.java | 23 +- .../persistence/model/entity/EntityMap.java | 66 ++ .../model/entity/EntityToMapConverter.java | 133 ++++ .../model/entity/MapToEntityConverter.java | 203 ++++++ .../persistence/model/entity/SimpleId.java | 1 - .../persistence/model/field/AbstractField.java | 8 + .../persistence/model/field/ListField.java | 2 +- .../persistence/model/field/StringField.java | 20 + .../usergrid/persistence/index/EntityIndex.java | 4 +- .../index/impl/EsEntityIndexImpl.java | 96 +-- .../index/guice/TestIndexModule.java | 7 +- .../index/impl/EntityIndexMapUtils.java | 52 +- .../queue/guice/TestQueueModule.java | 3 +- stack/pom.xml | 2 +- .../org/apache/usergrid/rest/IndexResource.java | 3 + .../apache/usergrid/rest/MigrateResource.java | 94 ++- .../applications/ApplicationsResource.java | 6 - 232 files changed, 9243 insertions(+), 5174 deletions(-) ----------------------------------------------------------------------