Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A295C200C56 for ; Fri, 14 Apr 2017 17:13:24 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A14EC160B80; Fri, 14 Apr 2017 15:13:24 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4742F160BB1 for ; Fri, 14 Apr 2017 17:13:20 +0200 (CEST) Received: (qmail 61795 invoked by uid 500); 14 Apr 2017 15:13:19 -0000 Mailing-List: contact commits-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list commits@ignite.apache.org Received: (qmail 60868 invoked by uid 99); 14 Apr 2017 15:13:19 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Apr 2017 15:13:19 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id EDFFDDFFAB; Fri, 14 Apr 2017 15:13:18 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: agura@apache.org To: commits@ignite.apache.org Date: Fri, 14 Apr 2017 15:14:03 -0000 Message-Id: <67c87c5871e64681a2cb468df9572db2@git.apache.org> In-Reply-To: <16dfbc567383453393a9f4c479744750@git.apache.org> References: <16dfbc567383453393a9f4c479744750@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [46/50] [abbrv] ignite git commit: IGNITE-4938: Moved OptimizedMarshaller to private package. This closes #1793. archived-at: Fri, 14 Apr 2017 15:13:24 -0000 IGNITE-4938: Moved OptimizedMarshaller to private package. This closes #1793. Project: http://git-wip-us.apache.org/repos/asf/ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/a1b5b8c3 Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/a1b5b8c3 Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/a1b5b8c3 Branch: refs/heads/ignite-4587 Commit: a1b5b8c39c847b308c3180bc5308b0a09adf75ec Parents: b3b5395 Author: Alexander Paschenko Authored: Fri Apr 14 14:21:57 2017 +0300 Committer: devozerov Committed: Fri Apr 14 14:21:58 2017 +0300 ---------------------------------------------------------------------- examples/config/filesystem/example-igfs.xml | 7 - .../MemcacheRestExampleNodeStartup.java | 7 - .../optimized/OptimizedMarshallerAopTest.java | 105 + .../optimized/OptimizedMarshallerAopTest.java | 104 - modules/clients/src/test/config/jdbc-config.xml | 2 +- .../apache/ignite/IgniteSystemProperties.java | 2 +- .../apache/ignite/internal/IgniteKernal.java | 2 +- .../internal/binary/BinaryClassDescriptor.java | 2 +- .../ignite/internal/binary/BinaryContext.java | 2 +- .../GridClientOptimizedMarshaller.java | 2 +- .../GridClientZipOptimizedMarshaller.java | 2 +- .../optimized/OptimizedClassDescriptor.java | 1141 +++++++++ .../optimized/OptimizedFieldType.java | 50 + .../optimized/OptimizedMarshaller.java | 313 +++ .../optimized/OptimizedMarshallerIdMapper.java | 33 + .../optimized/OptimizedMarshallerUtils.java | 551 +++++ .../optimized/OptimizedObjectInputStream.java | 1231 ++++++++++ .../optimized/OptimizedObjectOutputStream.java | 875 +++++++ .../OptimizedObjectStreamRegistry.java | 244 ++ .../apache/ignite/marshaller/Marshaller.java | 3 +- .../ignite/marshaller/jdk/JdkMarshaller.java | 3 +- .../optimized/OptimizedClassDescriptor.java | 1141 --------- .../optimized/OptimizedFieldType.java | 50 - .../optimized/OptimizedMarshaller.java | 313 --- .../optimized/OptimizedMarshallerIdMapper.java | 33 - .../optimized/OptimizedMarshallerUtils.java | 551 ----- .../optimized/OptimizedObjectInputStream.java | 1231 ---------- .../optimized/OptimizedObjectOutputStream.java | 875 ------- .../OptimizedObjectStreamRegistry.java | 244 -- .../marshaller/optimized/package-info.java | 22 - .../resources/META-INF/classnames.properties | 2 +- modules/core/src/test/config/example-cache.xml | 2 +- modules/core/src/test/config/igfs-loopback.xml | 2 +- modules/core/src/test/config/igfs-shmem.xml | 2 +- .../src/test/config/spring-start-nodes-attr.xml | 2 +- .../core/src/test/config/spring-start-nodes.xml | 2 +- .../config/websession/example-cache-base.xml | 2 +- .../IgniteExternalizableAbstractTest.java | 2 +- ...dbcPojoStoreOptimizedMarshallerSelfTest.java | 2 +- .../apache/ignite/igfs/IgfsPathSelfTest.java | 2 +- .../internal/GridLifecycleAwareSelfTest.java | 2 +- .../managers/GridManagerStopSelfTest.java | 2 +- .../GridDiscoveryManagerAttributesSelfTest.java | 2 +- .../OptimizedMarshallerEnumSelfTest.java | 87 + .../OptimizedMarshallerNodeFailoverTest.java | 358 +++ .../OptimizedMarshallerPooledSelfTest.java | 46 + .../optimized/OptimizedMarshallerSelfTest.java | 284 +++ ...arshallerSerialPersistentFieldsSelfTest.java | 116 + .../optimized/OptimizedMarshallerTest.java | 790 +++++++ .../OptimizedObjectStreamSelfTest.java | 2162 ++++++++++++++++++ .../TestTcpDiscoveryIpFinderAdapter.java | 43 + .../CacheStartupInDeploymentModesTest.java | 2 +- .../cache/GridCacheEntryMemorySizeSelfTest.java | 2 +- ...ridCacheStoreManagerDeserializationTest.java | 2 +- .../cache/GridCacheVersionSelfTest.java | 2 +- .../distributed/CacheAffinityEarlyTest.java | 2 +- .../CacheGetFutureHangsSelfTest.java | 2 +- .../IgniteCacheCreatePutMultiNodeSelfTest.java | 2 +- .../distributed/IgniteCacheCreatePutTest.java | 2 +- .../near/GridCacheNearTxForceKeyTest.java | 1 - .../CacheEntryProcessorNonSerializableTest.java | 2 +- .../igfs/IgfsAbstractBaseSelfTest.java | 2 +- .../processors/igfs/IgfsFileInfoSelfTest.java | 2 +- .../service/ClosureServiceClientsNodesTest.java | 2 +- ...ent2ClassLoadersOptimizedMarshallerTest.java | 2 +- ...mentClassLoadingOptimizedMarshallerTest.java | 2 +- ...namicProxySerializationMultiJvmSelfTest.java | 2 +- .../GridMarshallerPerformanceTest.java | 2 +- .../OptimizedMarshallerEnumSelfTest.java | 87 - .../OptimizedMarshallerNodeFailoverTest.java | 357 --- .../OptimizedMarshallerPooledSelfTest.java | 44 - .../optimized/OptimizedMarshallerSelfTest.java | 283 --- ...arshallerSerialPersistentFieldsSelfTest.java | 114 - .../optimized/OptimizedMarshallerTest.java | 790 ------- .../OptimizedObjectStreamSelfTest.java | 2157 ----------------- .../TestTcpDiscoveryIpFinderAdapter.java | 43 - .../marshaller/optimized/package-info.java | 22 - .../IgniteMessagingWithClientTest.java | 2 +- ...pClientDiscoveryMarshallerCheckSelfTest.java | 2 +- .../TcpDiscoveryMarshallerCheckSelfTest.java | 2 +- .../configvariations/ConfigVariations.java | 2 +- .../junits/IgniteTestResources.java | 2 +- .../testsuites/IgniteBinaryBasicTestSuite.java | 14 +- .../IgniteMarshallerSelfTestSuite.java | 14 +- .../webapp/META-INF/ignite-webapp-config.xml | 2 +- .../test/config/hadoop-fs-open-test/grid-0.xml | 2 +- .../test/config/hadoop-fs-open-test/grid-1.xml | 2 +- .../test/config/hadoop-fs-open-test/grid-2.xml | 2 +- .../test/config/igfs-cli-config-dual-async.xml | 2 +- .../test/config/igfs-cli-config-dual-sync.xml | 2 +- .../src/test/config/igfs-cli-config-primary.xml | 2 +- .../src/test/config/igfs-cli-config-proxy.xml | 2 +- .../IgniteHadoopFileSystemAbstractSelfTest.java | 2 +- .../cache/IgniteCacheNoClassQuerySelfTest.java | 2 +- .../h2/sql/AbstractH2CompareQueryTest.java | 2 +- .../FetchingQueryCursorStressTest.java | 2 +- .../Config/marshaller-invalid.xml | 2 +- .../GridSpringBeanSerializationSelfTest.java | 2 +- .../generator/ConfigurationGenerator.js | 2 +- .../webapp2/META-INF/ignite-webapp-config.xml | 2 +- .../ignite-int-max-values-onheap-config.xml | 2 +- modules/yardstick/config/ignite-jdbc-config.xml | 2 +- 102 files changed, 8506 insertions(+), 8551 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/examples/config/filesystem/example-igfs.xml ---------------------------------------------------------------------- diff --git a/examples/config/filesystem/example-igfs.xml b/examples/config/filesystem/example-igfs.xml index 2236821..454dc9f 100644 --- a/examples/config/filesystem/example-igfs.xml +++ b/examples/config/filesystem/example-igfs.xml @@ -50,13 +50,6 @@ Configuration below demonstrates how to setup a IgniteFs node with file data. --> - - - - - - - http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/examples/src/main/java/org/apache/ignite/examples/misc/client/memcache/MemcacheRestExampleNodeStartup.java ---------------------------------------------------------------------- diff --git a/examples/src/main/java/org/apache/ignite/examples/misc/client/memcache/MemcacheRestExampleNodeStartup.java b/examples/src/main/java/org/apache/ignite/examples/misc/client/memcache/MemcacheRestExampleNodeStartup.java index 3c835be..439e042 100644 --- a/examples/src/main/java/org/apache/ignite/examples/misc/client/memcache/MemcacheRestExampleNodeStartup.java +++ b/examples/src/main/java/org/apache/ignite/examples/misc/client/memcache/MemcacheRestExampleNodeStartup.java @@ -23,7 +23,6 @@ import org.apache.ignite.Ignition; import org.apache.ignite.configuration.CacheConfiguration; import org.apache.ignite.configuration.ConnectorConfiguration; import org.apache.ignite.configuration.IgniteConfiguration; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi; import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder; @@ -64,12 +63,6 @@ public class MemcacheRestExampleNodeStartup { cfg.setConnectorConfiguration(new ConnectorConfiguration()); - OptimizedMarshaller marsh = new OptimizedMarshaller(); - - marsh.setRequireSerializable(false); - - cfg.setMarshaller(marsh); - CacheConfiguration cacheCfg = new CacheConfiguration(); cacheCfg.setAtomicityMode(TRANSACTIONAL); http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/aop/src/test/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerAopTest.java ---------------------------------------------------------------------- diff --git a/modules/aop/src/test/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerAopTest.java b/modules/aop/src/test/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerAopTest.java new file mode 100644 index 0000000..a1c52ab --- /dev/null +++ b/modules/aop/src/test/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerAopTest.java @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.marshaller.optimized; + +import java.util.concurrent.atomic.AtomicInteger; +import org.apache.ignite.compute.gridify.Gridify; +import org.apache.ignite.configuration.IgniteConfiguration; +import org.apache.ignite.events.Event; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.util.typedef.G; +import org.apache.ignite.internal.util.typedef.X; +import org.apache.ignite.lang.IgnitePredicate; +import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest; + +import static org.apache.ignite.events.EventType.EVT_TASK_FINISHED; + +/** + * Test use GridOptimizedMarshaller and AspectJ AOP. + * + * The following configuration needs to be applied to enable AspectJ byte code + * weaving. + *
    + *
  • + * JVM configuration should include: + * -javaagent:[IGNITE_HOME]/libs/aspectjweaver-1.7.2.jar + *
    • + *
  • + * Classpath should contain the [IGNITE_HOME]/modules/tests/config/aop/aspectj folder. + *
    • + *
+ */ +public class OptimizedMarshallerAopTest extends GridCommonAbstractTest { + /** */ + private static final AtomicInteger cntr = new AtomicInteger(); + + /** + * Constructs a test. + */ + public OptimizedMarshallerAopTest() { + super(false /* start grid. */); + } + + /** {@inheritDoc} */ + @Override protected void beforeTestsStarted() throws Exception { + IgniteConfiguration cfg = new IgniteConfiguration(); + + cfg.setMarshaller(new OptimizedMarshaller()); + + G.start(cfg); + + assert G.allGrids().size() == 1; + } + + /** {@inheritDoc} */ + @Override protected void afterTestsStopped() throws Exception { + stopAllGrids(); + + assert G.allGrids().isEmpty(); + } + + /** + * JUnit. + * + * @throws Exception If failed. + */ + public void testUp() throws Exception { + G.ignite().events().localListen(new IgnitePredicate() { + @Override public boolean apply(Event evt) { + cntr.incrementAndGet(); + + return true; + } + }, EVT_TASK_FINISHED); + + gridify1(); + + assertEquals("Method gridify() wasn't executed on grid.", 1, cntr.get()); + } + + /** + * Method grid-enabled with {@link org.apache.ignite.compute.gridify.Gridify} annotation. + *

+ * Note that default {@code Gridify} configuration is used, so this method + * will be executed on remote node with the same argument. + */ + @Gridify + private void gridify1() { + X.println("Executes on grid"); + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/aop/src/test/java/org/apache/ignite/marshaller/optimized/OptimizedMarshallerAopTest.java ---------------------------------------------------------------------- diff --git a/modules/aop/src/test/java/org/apache/ignite/marshaller/optimized/OptimizedMarshallerAopTest.java b/modules/aop/src/test/java/org/apache/ignite/marshaller/optimized/OptimizedMarshallerAopTest.java deleted file mode 100644 index 7985fb2..0000000 --- a/modules/aop/src/test/java/org/apache/ignite/marshaller/optimized/OptimizedMarshallerAopTest.java +++ /dev/null @@ -1,104 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.ignite.marshaller.optimized; - -import java.util.concurrent.atomic.AtomicInteger; -import org.apache.ignite.compute.gridify.Gridify; -import org.apache.ignite.configuration.IgniteConfiguration; -import org.apache.ignite.events.Event; -import org.apache.ignite.internal.util.typedef.G; -import org.apache.ignite.internal.util.typedef.X; -import org.apache.ignite.lang.IgnitePredicate; -import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest; - -import static org.apache.ignite.events.EventType.EVT_TASK_FINISHED; - -/** - * Test use GridOptimizedMarshaller and AspectJ AOP. - * - * The following configuration needs to be applied to enable AspectJ byte code - * weaving. - *

    - *
  • - * JVM configuration should include: - * -javaagent:[IGNITE_HOME]/libs/aspectjweaver-1.7.2.jar - *
    • - *
  • - * Classpath should contain the [IGNITE_HOME]/modules/tests/config/aop/aspectj folder. - *
    • - *
- */ -public class OptimizedMarshallerAopTest extends GridCommonAbstractTest { - /** */ - private static final AtomicInteger cntr = new AtomicInteger(); - - /** - * Constructs a test. - */ - public OptimizedMarshallerAopTest() { - super(false /* start grid. */); - } - - /** {@inheritDoc} */ - @Override protected void beforeTestsStarted() throws Exception { - IgniteConfiguration cfg = new IgniteConfiguration(); - - cfg.setMarshaller(new OptimizedMarshaller()); - - G.start(cfg); - - assert G.allGrids().size() == 1; - } - - /** {@inheritDoc} */ - @Override protected void afterTestsStopped() throws Exception { - stopAllGrids(); - - assert G.allGrids().isEmpty(); - } - - /** - * JUnit. - * - * @throws Exception If failed. - */ - public void testUp() throws Exception { - G.ignite().events().localListen(new IgnitePredicate() { - @Override public boolean apply(Event evt) { - cntr.incrementAndGet(); - - return true; - } - }, EVT_TASK_FINISHED); - - gridify1(); - - assertEquals("Method gridify() wasn't executed on grid.", 1, cntr.get()); - } - - /** - * Method grid-enabled with {@link org.apache.ignite.compute.gridify.Gridify} annotation. - *

- * Note that default {@code Gridify} configuration is used, so this method - * will be executed on remote node with the same argument. - */ - @Gridify - private void gridify1() { - X.println("Executes on grid"); - } -} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/clients/src/test/config/jdbc-config.xml ---------------------------------------------------------------------- diff --git a/modules/clients/src/test/config/jdbc-config.xml b/modules/clients/src/test/config/jdbc-config.xml index 27d2617..7bba410 100644 --- a/modules/clients/src/test/config/jdbc-config.xml +++ b/modules/clients/src/test/config/jdbc-config.xml @@ -32,7 +32,7 @@ - + http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java b/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java index 0b4644f..1216db8 100644 --- a/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java +++ b/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java @@ -24,7 +24,7 @@ import java.util.Map; import java.util.Properties; import javax.net.ssl.HostnameVerifier; import org.apache.ignite.lang.IgnitePredicate; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; import org.jetbrains.annotations.Nullable; /** http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java b/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java index 286ced0..c991319 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java @@ -170,7 +170,7 @@ import org.apache.ignite.lifecycle.LifecycleAware; import org.apache.ignite.lifecycle.LifecycleBean; import org.apache.ignite.lifecycle.LifecycleEventType; import org.apache.ignite.marshaller.MarshallerExclusions; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; import org.apache.ignite.mxbean.ClusterLocalNodeMetricsMXBean; import org.apache.ignite.mxbean.IgniteMXBean; import org.apache.ignite.mxbean.StripedExecutorMXBean; http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryClassDescriptor.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryClassDescriptor.java b/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryClassDescriptor.java index 92e4874..f6e637f 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryClassDescriptor.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryClassDescriptor.java @@ -48,7 +48,7 @@ import org.apache.ignite.internal.util.typedef.F; import org.apache.ignite.internal.util.typedef.internal.S; import org.apache.ignite.internal.util.typedef.internal.U; import org.apache.ignite.marshaller.MarshallerExclusions; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; import org.jetbrains.annotations.Nullable; import static org.apache.ignite.internal.processors.query.QueryUtils.isGeometryClass; http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryContext.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryContext.java b/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryContext.java index 1ce2d65..f339253 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryContext.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryContext.java @@ -87,7 +87,7 @@ import org.apache.ignite.lang.IgniteBiTuple; import org.apache.ignite.lang.IgniteUuid; import org.apache.ignite.marshaller.MarshallerContext; import org.apache.ignite.marshaller.MarshallerUtils; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; import org.jetbrains.annotations.Nullable; import org.jsr166.ConcurrentHashMap8; http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientOptimizedMarshaller.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientOptimizedMarshaller.java b/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientOptimizedMarshaller.java index 6a4b5e8..75f1807 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientOptimizedMarshaller.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientOptimizedMarshaller.java @@ -26,7 +26,7 @@ import org.apache.ignite.internal.MarshallerContextImpl; import org.apache.ignite.internal.client.marshaller.GridClientMarshaller; import org.apache.ignite.internal.processors.rest.client.message.GridClientMessage; import org.apache.ignite.internal.util.typedef.internal.U; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; import org.apache.ignite.plugin.PluginProvider; import org.jetbrains.annotations.Nullable; http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientZipOptimizedMarshaller.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientZipOptimizedMarshaller.java b/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientZipOptimizedMarshaller.java index b709634..bc8220f 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientZipOptimizedMarshaller.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/client/marshaller/optimized/GridClientZipOptimizedMarshaller.java @@ -31,7 +31,7 @@ import org.apache.ignite.IgniteCheckedException; import org.apache.ignite.internal.client.marshaller.GridClientMarshaller; import org.apache.ignite.internal.processors.rest.client.message.GridClientMessage; import org.apache.ignite.internal.util.typedef.internal.U; -import org.apache.ignite.marshaller.optimized.OptimizedMarshaller; +import org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller; import org.apache.ignite.plugin.PluginProvider; import org.jetbrains.annotations.Nullable; http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedClassDescriptor.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedClassDescriptor.java b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedClassDescriptor.java new file mode 100644 index 0000000..4c0ce17 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedClassDescriptor.java @@ -0,0 +1,1141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.marshaller.optimized; + +import java.io.Externalizable; +import java.io.IOException; +import java.io.NotSerializableException; +import java.io.ObjectInputStream; +import java.io.ObjectOutputStream; +import java.io.ObjectStreamField; +import java.io.Serializable; +import java.lang.reflect.Constructor; +import java.lang.reflect.Field; +import java.lang.reflect.InvocationHandler; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.lang.reflect.Proxy; +import java.util.ArrayList; +import java.util.Collections; +import java.util.Comparator; +import java.util.Date; +import java.util.HashMap; +import java.util.HashSet; +import java.util.LinkedHashMap; +import java.util.LinkedHashSet; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import java.util.UUID; +import java.util.concurrent.ConcurrentMap; + +import org.apache.ignite.internal.util.GridUnsafe; +import org.apache.ignite.internal.util.typedef.internal.U; +import org.apache.ignite.lang.IgniteProductVersion; +import org.apache.ignite.marshaller.MarshallerContext; +import org.apache.ignite.marshaller.MarshallerExclusions; +import org.apache.ignite.internal.util.SerializableTransient; +import org.apache.ignite.marshaller.MarshallerUtils; + +import static java.lang.reflect.Modifier.isFinal; +import static java.lang.reflect.Modifier.isPrivate; +import static java.lang.reflect.Modifier.isStatic; +import static java.lang.reflect.Modifier.isTransient; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.ARRAY_LIST; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.BOOLEAN; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.BOOLEAN_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.BYTE; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.BYTE_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.CHAR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.CHAR_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.CLS; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.DATE; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.DOUBLE; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.DOUBLE_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.ENUM; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.EXTERNALIZABLE; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.FLOAT; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.FLOAT_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.HASH_MAP; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.HASH_SET; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.HASH_SET_MAP_OFF; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.INT; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.INT_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.LINKED_HASH_MAP; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.LINKED_HASH_SET; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.LINKED_LIST; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.LONG; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.LONG_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.OBJ_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.PROPS; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.PROXY; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.SERIALIZABLE; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.SHORT; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.SHORT_ARR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.STR; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.UUID; +import static org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.computeSerialVersionUid; + +/** + * Class descriptor. + */ +class OptimizedClassDescriptor { + /** Class. */ + private final Class cls; + + /** Context. */ + private final MarshallerContext ctx; + + /** */ + private ConcurrentMap clsMap; + + /** ID mapper. */ + private final OptimizedMarshallerIdMapper mapper; + + /** Class name. */ + private final String name; + + /** Type ID. */ + private final int typeId; + + /** Short ID. */ + private final short checksum; + + /** Class type. */ + private int type; + + /** Primitive flag. */ + private boolean isPrimitive; + + /** Enum flag. */ + private boolean isEnum; + + /** Serializable flag. */ + private boolean isSerial; + + /** Excluded flag. */ + private boolean excluded; + + /** {@code True} if descriptor is for {@link Class}. */ + private boolean isCls; + + /** Enumeration values. */ + private Object[] enumVals; + + /** Constructor. */ + private Constructor constructor; + + /** Fields. */ + private Fields fields; + + /** {@code writeObject} methods. */ + private List writeObjMtds; + + /** {@code writeReplace} method. */ + private Method writeReplaceMtd; + + /** {@code readObject} methods. */ + private List readObjMtds; + + /** {@code readResolve} method. */ + private Method readResolveMtd; + + /** Defaults field offset. */ + private long dfltsFieldOff; + + /** Load factor field offset. */ + private long loadFactorFieldOff; + + /** Access order field offset. */ + private long accessOrderFieldOff; + + /** Proxy interfaces. */ + private Class[] proxyIntfs; + + /** Method returns serializable transient fields. */ + private Method serTransMtd; + + /** + * Creates descriptor for class. + * + * @param typeId Type ID. + * @param clsMap Class descriptors by class map. + * @param cls Class. + * @param ctx Context. + * @param mapper ID mapper. + * @throws IOException In case of error. + */ + @SuppressWarnings("ForLoopReplaceableByForEach") + OptimizedClassDescriptor(Class cls, + int typeId, + ConcurrentMap clsMap, + MarshallerContext ctx, + OptimizedMarshallerIdMapper mapper) + throws IOException { + this.cls = cls; + this.typeId = typeId; + this.clsMap = clsMap; + this.ctx = ctx; + this.mapper = mapper; + + name = cls.getName(); + + excluded = MarshallerExclusions.isExcluded(cls); + + if (!excluded) { + Class parent; + + if (cls == byte.class || cls == Byte.class) { + type = BYTE; + + isPrimitive = true; + } + else if (cls == short.class || cls == Short.class) { + type = SHORT; + + isPrimitive = true; + } + else if (cls == int.class || cls == Integer.class) { + type = INT; + + isPrimitive = true; + } + else if (cls == long.class || cls == Long.class) { + type = LONG; + + isPrimitive = true; + } + else if (cls == float.class || cls == Float.class) { + type = FLOAT; + + isPrimitive = true; + } + else if (cls == double.class || cls == Double.class) { + type = DOUBLE; + + isPrimitive = true; + } + else if (cls == char.class || cls == Character.class) { + type = CHAR; + + isPrimitive = true; + } + else if (cls == boolean.class || cls == Boolean.class) { + type = BOOLEAN; + + isPrimitive = true; + } + else if (cls == byte[].class) + type = BYTE_ARR; + else if (cls == short[].class) + type = SHORT_ARR; + else if (cls == int[].class) + type = INT_ARR; + else if (cls == long[].class) + type = LONG_ARR; + else if (cls == float[].class) + type = FLOAT_ARR; + else if (cls == double[].class) + type = DOUBLE_ARR; + else if (cls == char[].class) + type = CHAR_ARR; + else if (cls == boolean[].class) + type = BOOLEAN_ARR; + else if (cls.isArray()) + type = OBJ_ARR; + else if (cls == String.class) + type = STR; + else if (cls.isEnum()) { + type = ENUM; + + isEnum = true; + enumVals = cls.getEnumConstants(); + } + // Support for enum constants, based on anonymous children classes. + else if ((parent = cls.getSuperclass()) != null && parent.isEnum()) { + type = ENUM; + + isEnum = true; + enumVals = parent.getEnumConstants(); + } + else if (cls == UUID.class) + type = UUID; + else if (cls == Properties.class) { + type = PROPS; + + try { + dfltsFieldOff = GridUnsafe.objectFieldOffset(Properties.class.getDeclaredField("defaults")); + } + catch (NoSuchFieldException e) { + throw new IOException(e); + } + } + else if (cls == ArrayList.class) + type = ARRAY_LIST; + else if (cls == HashMap.class) { + type = HASH_MAP; + + try { + loadFactorFieldOff = GridUnsafe.objectFieldOffset(HashMap.class.getDeclaredField("loadFactor")); + } + catch (NoSuchFieldException e) { + throw new IOException(e); + } + } + else if (cls == HashSet.class) { + type = HASH_SET; + + try { + loadFactorFieldOff = GridUnsafe.objectFieldOffset(HashMap.class.getDeclaredField("loadFactor")); + } + catch (NoSuchFieldException e) { + throw new IOException(e); + } + } + else if (cls == LinkedList.class) + type = LINKED_LIST; + else if (cls == LinkedHashMap.class) { + type = LINKED_HASH_MAP; + + try { + loadFactorFieldOff = + GridUnsafe.objectFieldOffset(HashMap.class.getDeclaredField("loadFactor")); + accessOrderFieldOff = + GridUnsafe.objectFieldOffset(LinkedHashMap.class.getDeclaredField("accessOrder")); + } + catch (NoSuchFieldException e) { + throw new IOException(e); + } + } + else if (cls == LinkedHashSet.class) { + type = LINKED_HASH_SET; + + try { + loadFactorFieldOff = GridUnsafe.objectFieldOffset(HashMap.class.getDeclaredField("loadFactor")); + } + catch (NoSuchFieldException e) { + throw new IOException(e); + } + } + else if (cls == Date.class) + type = DATE; + else if (cls == Class.class) { + type = CLS; + + isCls = true; + } + else if (Proxy.class.isAssignableFrom(cls)) { + type = PROXY; + + proxyIntfs = cls.getInterfaces(); + } + else { + Class c = cls; + + while ((writeReplaceMtd == null || readResolveMtd == null) && c != null && !c.equals(Object.class)) { + if (writeReplaceMtd == null) { + try { + writeReplaceMtd = c.getDeclaredMethod("writeReplace"); + + if (!isStatic(writeReplaceMtd.getModifiers()) && + !(isPrivate(writeReplaceMtd.getModifiers()) && c != cls) && + writeReplaceMtd.getReturnType().equals(Object.class)) + writeReplaceMtd.setAccessible(true); + else + // Set method back to null if it has incorrect signature. + writeReplaceMtd = null; + } + catch (NoSuchMethodException ignored) { + // No-op. + } + } + + if (readResolveMtd == null) { + try { + readResolveMtd = c.getDeclaredMethod("readResolve"); + + if (!isStatic(readResolveMtd.getModifiers()) && + !(isPrivate(readResolveMtd.getModifiers()) && c != cls) && + readResolveMtd.getReturnType().equals(Object.class)) + readResolveMtd.setAccessible(true); + else + // Set method back to null if it has incorrect signature. + readResolveMtd = null; + } + catch (NoSuchMethodException ignored) { + // No-op. + } + } + + c = c.getSuperclass(); + } + + if (Externalizable.class.isAssignableFrom(cls)) { + type = EXTERNALIZABLE; + + try { + constructor = !Modifier.isStatic(cls.getModifiers()) && cls.getDeclaringClass() != null ? + cls.getDeclaredConstructor(cls.getDeclaringClass()) : + cls.getDeclaredConstructor(); + + constructor.setAccessible(true); + } + catch (NoSuchMethodException e) { + throw new IOException("Externalizable class doesn't have default constructor: " + cls, e); + } + } + else { + type = SERIALIZABLE; + + isSerial = Serializable.class.isAssignableFrom(cls); + + writeObjMtds = new ArrayList<>(); + readObjMtds = new ArrayList<>(); + List fields = new ArrayList<>(); + + for (c = cls; c != null && !c.equals(Object.class); c = c.getSuperclass()) { + Method mtd; + + try { + mtd = c.getDeclaredMethod("writeObject", ObjectOutputStream.class); + + int mod = mtd.getModifiers(); + + if (!isStatic(mod) && isPrivate(mod) && mtd.getReturnType() == Void.TYPE) + mtd.setAccessible(true); + else + // Set method back to null if it has incorrect signature. + mtd = null; + } + catch (NoSuchMethodException ignored) { + mtd = null; + } + + writeObjMtds.add(mtd); + + try { + mtd = c.getDeclaredMethod("readObject", ObjectInputStream.class); + + int mod = mtd.getModifiers(); + + if (!isStatic(mod) && isPrivate(mod) && mtd.getReturnType() == Void.TYPE) + mtd.setAccessible(true); + else + // Set method back to null if it has incorrect signature. + mtd = null; + } + catch (NoSuchMethodException ignored) { + mtd = null; + } + + readObjMtds.add(mtd); + + final SerializableTransient serTransAn = c.getAnnotation(SerializableTransient.class); + + // Custom serialization policy for transient fields. + if (serTransAn != null) { + try { + serTransMtd = c.getDeclaredMethod(serTransAn.methodName(), cls, IgniteProductVersion.class); + + int mod = serTransMtd.getModifiers(); + + if (isStatic(mod) && isPrivate(mod) + && serTransMtd.getReturnType() == String[].class) + serTransMtd.setAccessible(true); + else + // Set method back to null if it has incorrect signature. + serTransMtd = null; + } + catch (NoSuchMethodException ignored) { + serTransMtd = null; + } + } + + Field[] clsFields0 = c.getDeclaredFields(); + + Map fieldNames = new HashMap<>(); + + for (Field f : clsFields0) + fieldNames.put(f.getName(), f); + + List clsFields = new ArrayList<>(clsFields0.length); + + boolean hasSerialPersistentFields = false; + + try { + Field serFieldsDesc = c.getDeclaredField("serialPersistentFields"); + + int mod = serFieldsDesc.getModifiers(); + + if (serFieldsDesc.getType() == ObjectStreamField[].class && + isPrivate(mod) && isStatic(mod) && isFinal(mod)) { + hasSerialPersistentFields = true; + + serFieldsDesc.setAccessible(true); + + ObjectStreamField[] serFields = (ObjectStreamField[]) serFieldsDesc.get(null); + + for (int i = 0; i < serFields.length; i++) { + ObjectStreamField serField = serFields[i]; + + FieldInfo fieldInfo; + + if (!fieldNames.containsKey(serField.getName())) { + fieldInfo = new FieldInfo(null, + serField.getName(), + -1, + fieldType(serField.getType())); + } + else { + Field f = fieldNames.get(serField.getName()); + + fieldInfo = new FieldInfo(f, + serField.getName(), + GridUnsafe.objectFieldOffset(f), + fieldType(serField.getType())); + } + + clsFields.add(fieldInfo); + } + } + } + catch (NoSuchFieldException ignored) { + // No-op. + } + catch (IllegalAccessException e) { + throw new IOException("Failed to get value of 'serialPersistentFields' field in class: " + + cls.getName(), e); + } + + if (!hasSerialPersistentFields) { + for (int i = 0; i < clsFields0.length; i++) { + Field f = clsFields0[i]; + + int mod = f.getModifiers(); + + if (!isStatic(mod) && !isTransient(mod)) { + FieldInfo fieldInfo = new FieldInfo(f, f.getName(), + GridUnsafe.objectFieldOffset(f), fieldType(f.getType())); + + clsFields.add(fieldInfo); + } + } + } + + Collections.sort(clsFields, new Comparator() { + @Override public int compare(FieldInfo t1, FieldInfo t2) { + return t1.name().compareTo(t2.name()); + } + }); + + fields.add(new ClassFields(clsFields)); + } + + Collections.reverse(writeObjMtds); + Collections.reverse(readObjMtds); + Collections.reverse(fields); + + this.fields = new Fields(fields); + } + } + } + + checksum = computeSerialVersionUid(cls, fields != null ? fields.ownFields() : null); + } + + /** + * @return Excluded flag. + */ + boolean excluded() { + return excluded; + } + + /** + * @return Class. + */ + Class describedClass() { + return cls; + } + + /** + * @return Primitive flag. + */ + boolean isPrimitive() { + return isPrimitive; + } + + /** + * @return Enum flag. + */ + boolean isEnum() { + return isEnum; + } + + /** + * @return {@code True} if descriptor is for {@link Class}. + */ + boolean isClass() { + return isCls; + } + + /** + * @return {@code True} if descriptor is for {@link Proxy}. + */ + boolean isProxy() { + return type == PROXY; + } + + /** + * Replaces object. + * + * @param obj Object. + * @return Replaced object or {@code null} if there is no {@code writeReplace} method. + * @throws IOException In case of error. + */ + Object replace(Object obj) throws IOException { + if (writeReplaceMtd != null) { + try { + return writeReplaceMtd.invoke(obj); + } + catch (IllegalAccessException | InvocationTargetException e) { + throw new IOException(e); + } + } + + return obj; + } + + /** + * Writes object to stream. + * + * @param out Output stream. + * @param obj Object. + * @throws IOException In case of error. + */ + @SuppressWarnings("ForLoopReplaceableByForEach") + void write(OptimizedObjectOutputStream out, Object obj) throws IOException { + out.write(type); + + switch (type) { + case BYTE: + out.writeByte((Byte)obj); + + break; + + case SHORT: + out.writeShort((Short)obj); + + break; + + case INT: + out.writeInt((Integer)obj); + + break; + + case LONG: + out.writeLong((Long)obj); + + break; + + case FLOAT: + out.writeFloat((Float)obj); + + break; + + case DOUBLE: + out.writeDouble((Double)obj); + + break; + + case CHAR: + out.writeChar((Character)obj); + + break; + + case BOOLEAN: + out.writeBoolean((Boolean)obj); + + break; + + case BYTE_ARR: + out.writeByteArray((byte[])obj); + + break; + + case SHORT_ARR: + out.writeShortArray((short[])obj); + + break; + + case INT_ARR: + out.writeIntArray((int[])obj); + + break; + + case LONG_ARR: + out.writeLongArray((long[])obj); + + break; + + case FLOAT_ARR: + out.writeFloatArray((float[])obj); + + break; + + case DOUBLE_ARR: + out.writeDoubleArray((double[])obj); + + break; + + case CHAR_ARR: + out.writeCharArray((char[])obj); + + break; + + case BOOLEAN_ARR: + out.writeBooleanArray((boolean[])obj); + + break; + + case OBJ_ARR: + OptimizedClassDescriptor compDesc = OptimizedMarshallerUtils.classDescriptor(clsMap, + obj.getClass().getComponentType(), + ctx, + mapper); + + compDesc.writeTypeData(out); + + out.writeArray((Object[])obj); + + break; + + case STR: + out.writeString((String)obj); + + break; + + case UUID: + out.writeUuid((UUID)obj); + + break; + + case PROPS: + out.writeProperties((Properties)obj, dfltsFieldOff); + + break; + + case ARRAY_LIST: + out.writeArrayList((ArrayList)obj); + + break; + + case HASH_MAP: + out.writeHashMap((HashMap)obj, loadFactorFieldOff, false); + + break; + + case HASH_SET: + out.writeHashSet((HashSet)obj, HASH_SET_MAP_OFF, loadFactorFieldOff); + + break; + + case LINKED_LIST: + out.writeLinkedList((LinkedList)obj); + + break; + + case LINKED_HASH_MAP: + out.writeLinkedHashMap((LinkedHashMap)obj, loadFactorFieldOff, accessOrderFieldOff, false); + + break; + + case LINKED_HASH_SET: + out.writeLinkedHashSet((LinkedHashSet)obj, HASH_SET_MAP_OFF, loadFactorFieldOff); + + break; + + case DATE: + out.writeDate((Date)obj); + + break; + + case CLS: + OptimizedClassDescriptor clsDesc = OptimizedMarshallerUtils.classDescriptor(clsMap, (Class)obj, ctx, mapper); + + clsDesc.writeTypeData(out); + + break; + + case PROXY: + out.writeInt(proxyIntfs.length); + + for (Class intf : proxyIntfs) { + OptimizedClassDescriptor intfDesc = OptimizedMarshallerUtils.classDescriptor(clsMap, intf, ctx, mapper); + + intfDesc.writeTypeData(out); + } + + InvocationHandler ih = Proxy.getInvocationHandler(obj); + + assert ih != null; + + out.writeObject(ih); + + break; + + case ENUM: + writeTypeData(out); + + out.writeInt(((Enum)obj).ordinal()); + + break; + + case EXTERNALIZABLE: + writeTypeData(out); + + out.writeShort(checksum); + out.writeExternalizable(obj); + + break; + + case SERIALIZABLE: + if (out.requireSerializable() && !isSerial) + throw new NotSerializableException("Must implement java.io.Serializable or " + + "set OptimizedMarshaller.setRequireSerializable() to false " + + "(note that performance may degrade if object is not Serializable): " + name); + + writeTypeData(out); + + out.writeShort(checksum); + out.writeSerializable(obj, writeObjMtds, serializableFields(obj.getClass(), obj, null)); + + break; + + default: + throw new IllegalStateException("Invalid class type: " + type); + } + } + + /** + * Gets list of serializable fields. If {@link #serTransMtd} method + * returns list of transient fields, they will be added to other fields. + * Transient fields that are not included in that list will be normally + * ignored. + * + * @param cls Class. + * @param obj Object. + * @param ver Job sender version. + * @return Serializable fields. + */ + @SuppressWarnings("ForLoopReplaceableByForEach") + private Fields serializableFields(Class cls, Object obj, IgniteProductVersion ver) { + if (serTransMtd == null) + return fields; + + try { + final String[] transFields = (String[])serTransMtd.invoke(cls, obj, ver); + + if (transFields == null || transFields.length == 0) + return fields; + + List clsFields = new ArrayList<>(); + + clsFields.addAll(fields.fields.get(0).fields); + + for (int i = 0; i < transFields.length; i++) { + final String fieldName = transFields[i]; + + final Field f = cls.getDeclaredField(fieldName); + + FieldInfo fieldInfo = new FieldInfo(f, f.getName(), + GridUnsafe.objectFieldOffset(f), fieldType(f.getType())); + + clsFields.add(fieldInfo); + } + + Collections.sort(clsFields, new Comparator() { + @Override public int compare(FieldInfo t1, FieldInfo t2) { + return t1.name().compareTo(t2.name()); + } + }); + + List fields = new ArrayList<>(); + + fields.add(new ClassFields(clsFields)); + + return new Fields(fields); + } + catch (Exception ignored) { + return fields; + } + } + + /** + * @param out Output stream. + * @throws IOException In case of error. + */ + void writeTypeData(OptimizedObjectOutputStream out) throws IOException { + out.writeInt(typeId); + + if (typeId == 0) + out.writeUTF(name); + } + + /** + * Reads object from stream. + * + * @param in Input stream. + * @return Object. + * @throws ClassNotFoundException If class not found. + * @throws IOException In case of error. + */ + Object read(OptimizedObjectInputStream in) throws ClassNotFoundException, IOException { + switch (type) { + case ENUM: + return enumVals[in.readInt()]; + + case EXTERNALIZABLE: + verifyChecksum(in.readShort()); + + return in.readExternalizable(constructor, readResolveMtd); + + case SERIALIZABLE: + verifyChecksum(in.readShort()); + + // If no serialize method, then unmarshal as usual. + if (serTransMtd != null) + return in.readSerializable(cls, readObjMtds, readResolveMtd, + serializableFields(cls, null, MarshallerUtils.jobSenderVersion())); + else + return in.readSerializable(cls, readObjMtds, readResolveMtd, fields); + + default: + assert false : "Unexpected type: " + type; + + return null; + } + } + + /** + * @param checksum Checksum. + * @throws ClassNotFoundException If checksum is wrong. + * @throws IOException In case of error. + */ + private void verifyChecksum(short checksum) throws ClassNotFoundException, IOException { + if (checksum != this.checksum) + throw new ClassNotFoundException("Optimized stream class checksum mismatch " + + "(is same version of marshalled class present on all nodes?) " + + "[expected=" + this.checksum + ", actual=" + checksum + ", cls=" + cls + ']'); + } + + /** + * @param cls Class. + * @return Type. + */ + @SuppressWarnings("IfMayBeConditional") + private OptimizedFieldType fieldType(Class cls) { + OptimizedFieldType type; + + if (cls == byte.class) + type = OptimizedFieldType.BYTE; + else if (cls == short.class) + type = OptimizedFieldType.SHORT; + else if (cls == int.class) + type = OptimizedFieldType.INT; + else if (cls == long.class) + type = OptimizedFieldType.LONG; + else if (cls == float.class) + type = OptimizedFieldType.FLOAT; + else if (cls == double.class) + type = OptimizedFieldType.DOUBLE; + else if (cls == char.class) + type = OptimizedFieldType.CHAR; + else if (cls == boolean.class) + type = OptimizedFieldType.BOOLEAN; + else + type = OptimizedFieldType.OTHER; + + return type; + } + + /** + * Information about one field. + */ + @SuppressWarnings("PackageVisibleInnerClass") + static class FieldInfo { + /** Field. */ + private final Field field; + + /** Field offset. */ + private final long fieldOffs; + + /** Field type. */ + private final OptimizedFieldType fieldType; + + /** Field name. */ + private final String fieldName; + + /** + * @param field Field. + * @param name Field name. + * @param offset Field offset. + * @param type Grid optimized field type. + */ + FieldInfo(Field field, String name, long offset, OptimizedFieldType type) { + this.field = field; + fieldOffs = offset; + fieldType = type; + fieldName = name; + } + + /** + * @return Returns field. + */ + Field field() { + return field; + } + + /** + * @return Offset. + */ + long offset() { + return fieldOffs; + } + + /** + * @return Type. + */ + OptimizedFieldType type() { + return fieldType; + } + + /** + * @return Name. + */ + String name() { + return fieldName; + } + } + + /** + * Information about one class. + */ + static class ClassFields { + /** Fields. */ + private final List fields; + + private final Map nameToIndex; + + /** + * @param fields Field infos. + */ + ClassFields(List fields) { + this.fields = fields; + + nameToIndex = U.newHashMap(fields.size()); + + for (int i = 0; i < fields.size(); ++i) + nameToIndex.put(fields.get(i).name(), i); + } + + /** + * @return Class fields. + */ + List fields() { + return fields; + } + + /** + * @return Fields count. + */ + int size() { + return fields.size(); + } + + /** + * @param i Field's index. + * @return FieldInfo. + */ + FieldInfo get(int i) { + return fields.get(i); + } + + /** + * @param name Field's name. + * @return Field's index. + */ + int getIndex(String name) { + assert nameToIndex.containsKey(name); + + return nameToIndex.get(name); + } + } + + /** + * Encapsulates data about class fields. + */ + @SuppressWarnings("PackageVisibleInnerClass") + static class Fields { + /** Fields. */ + private final List fields; + + /** Own fields (excluding inherited). */ + private final List ownFields; + + /** + * Creates new instance. + * + * @param fields Fields. + */ + Fields(List fields) { + this.fields = fields; + + if (fields.isEmpty()) + ownFields = null; + else { + ownFields = new ArrayList<>(fields.size()); + + for (FieldInfo f : fields.get(fields.size() - 1).fields()) { + if (f.field() != null) + ownFields.add(f.field); + } + } + } + + /** + * Returns class's own fields (excluding inherited). + * + * @return List of fields or {@code null} if fields list is empty. + */ + List ownFields() { + return ownFields; + } + + /** + * Returns field types and their offsets. + * + * @param i hierarchy level where 0 corresponds to top level. + * @return list of pairs where first value is field type and second value is its offset. + */ + ClassFields fields(int i) { + return fields.get(i); + } + } +} http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedFieldType.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedFieldType.java b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedFieldType.java new file mode 100644 index 0000000..39cad13 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedFieldType.java @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.marshaller.optimized; + +/** + * Field type used to calculate {@code Unsafe} offsets into objects. + */ +enum OptimizedFieldType { + /** */ + BYTE, + + /** */ + SHORT, + + /** */ + INT, + + /** */ + LONG, + + /** */ + FLOAT, + + /** */ + DOUBLE, + + /** */ + CHAR, + + /** */ + BOOLEAN, + + /** */ + OTHER +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshaller.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshaller.java b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshaller.java new file mode 100644 index 0000000..3f40d63 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshaller.java @@ -0,0 +1,313 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.marshaller.optimized; + +import java.io.IOException; +import java.io.InputStream; +import java.io.ObjectOutputStream; +import java.io.OutputStream; +import java.io.Serializable; +import java.util.concurrent.ConcurrentMap; +import org.apache.ignite.IgniteCheckedException; +import org.apache.ignite.IgniteException; +import org.apache.ignite.IgniteSystemProperties; +import org.apache.ignite.internal.util.typedef.internal.U; +import org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller; +import org.jetbrains.annotations.Nullable; +import org.jsr166.ConcurrentHashMap8; +import sun.misc.Unsafe; + +import static org.apache.ignite.IgniteSystemProperties.IGNITE_OPTIMIZED_MARSHALLER_USE_DEFAULT_SUID; + +/** + * Optimized implementation of {@link org.apache.ignite.marshaller.Marshaller}. + * Unlike {@link org.apache.ignite.marshaller.jdk.JdkMarshaller}, + * which is based on standard {@link ObjectOutputStream}, this marshaller does not + * enforce that all serialized objects implement {@link Serializable} interface. It is also + * about 20 times faster as it removes lots of serialization overhead that exists in + * default JDK implementation. + *

+ * {@code OptimizedMarshaller} is tested only on Java HotSpot VM on other VMs + * it could yield unexpected results. It is the default marshaller on Java HotSpot VMs + * and will be used if no other marshaller was explicitly configured. + *

+ *

Configuration

+ *

Mandatory

+ * This marshaller has no mandatory configuration parameters. + *

Java Example

+ * 
+ * OptimizedMarshaller marshaller = new OptimizedMarshaller();
+ *
+ * // Enforce Serializable interface.
+ * marshaller.setRequireSerializable(true);
+ *
+ * IgniteConfiguration cfg = new IgniteConfiguration();
+ *
+ * // Override marshaller.
+ * cfg.setMarshaller(marshaller);
+ *
+ * // Starts grid.
+ * G.start(cfg);
+ *
+ *

Spring Example

+ * GridOptimizedMarshaller can be configured from Spring XML configuration file: + * 
+ * <bean id="grid.custom.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" singleton="true">
+ *     ...
+ *     <property name="marshaller">
+ *         <bean class="org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller">
+ *             <property name="requireSerializable">true</property>
+ *         </bean>
+ *     </property>
+ *     ...
+ * </bean>
+ *
+ *

+ * + *
+ * For information about Spring framework visit www.springframework.org + */ +public class OptimizedMarshaller extends AbstractNodeNameAwareMarshaller { + /** Use default {@code serialVersionUID} for {@link Serializable} classes. */ + public static final boolean USE_DFLT_SUID = + IgniteSystemProperties.getBoolean(IGNITE_OPTIMIZED_MARSHALLER_USE_DEFAULT_SUID, false); + + /** Default class loader. */ + private final ClassLoader dfltClsLdr = getClass().getClassLoader(); + + /** Whether or not to require an object to be serializable in order to be marshalled. */ + private boolean requireSer = true; + + /** ID mapper. */ + private OptimizedMarshallerIdMapper mapper; + + /** Class descriptors by class. */ + private final ConcurrentMap clsMap = new ConcurrentHashMap8<>(); + + /** + * Creates new marshaller will all defaults. + * + * @throws IgniteException If this marshaller is not supported on the current JVM. + */ + public OptimizedMarshaller() { + if (!available()) + throw new IgniteException("Using OptimizedMarshaller on unsupported JVM version (some of " + + "JVM-private APIs required for the marshaller to work are missing)."); + } + + /** + * Creates new marshaller providing whether it should + * require {@link Serializable} interface or not. + * + * @param requireSer Whether to require {@link Serializable}. + */ + public OptimizedMarshaller(boolean requireSer) { + this.requireSer = requireSer; + } + + /** + * Sets whether marshaller should require {@link Serializable} interface or not. + * + * @param requireSer Whether to require {@link Serializable}. + * @return {@code this} for chaining. + */ + public OptimizedMarshaller setRequireSerializable(boolean requireSer) { + this.requireSer = requireSer; + + return this; + } + + /** + * Sets ID mapper. + * + * @param mapper ID mapper. + * @return {@code this} for chaining. + */ + public OptimizedMarshaller setIdMapper(OptimizedMarshallerIdMapper mapper) { + this.mapper = mapper; + + return this; + } + + /** + * Specifies size of cached object streams used by marshaller. Object streams are cached for + * performance reason to avoid costly recreation for every serialization routine. If {@code 0} (default), + * pool is not used and each thread has its own cached object stream which it keeps reusing. + *

+ * Since each stream has an internal buffer, creating a stream for each thread can lead to + * high memory consumption if many large messages are marshalled or unmarshalled concurrently. + * Consider using pool in this case. This will limit number of streams that can be created and, + * therefore, decrease memory consumption. + *

+ * NOTE: Using streams pool can decrease performance since streams will be shared between + * different threads which will lead to more frequent context switching. + * + * @param poolSize Streams pool size. If {@code 0}, pool is not used. + * @return {@code this} for chaining. + */ + public OptimizedMarshaller setPoolSize(int poolSize) { + OptimizedObjectStreamRegistry.poolSize(poolSize); + + return this; + } + + /** {@inheritDoc} */ + @Override protected void marshal0(@Nullable Object obj, OutputStream out) throws IgniteCheckedException { + assert out != null; + + OptimizedObjectOutputStream objOut = null; + + try { + objOut = OptimizedObjectStreamRegistry.out(); + + objOut.context(clsMap, ctx, mapper, requireSer); + + objOut.out().outputStream(out); + + objOut.writeObject(obj); + } + catch (IOException e) { + throw new IgniteCheckedException("Failed to serialize object: " + obj, e); + } + finally { + OptimizedObjectStreamRegistry.closeOut(objOut); + } + } + + /** {@inheritDoc} */ + @Override protected byte[] marshal0(@Nullable Object obj) throws IgniteCheckedException { + OptimizedObjectOutputStream objOut = null; + + try { + objOut = OptimizedObjectStreamRegistry.out(); + + objOut.context(clsMap, ctx, mapper, requireSer); + + objOut.writeObject(obj); + + return objOut.out().array(); + } + catch (Exception e) { + throw new IgniteCheckedException("Failed to serialize object: " + obj, e); + } + finally { + OptimizedObjectStreamRegistry.closeOut(objOut); + } + } + + /** {@inheritDoc} */ + @SuppressWarnings("unchecked") + @Override protected T unmarshal0(InputStream in, @Nullable ClassLoader clsLdr) throws IgniteCheckedException { + assert in != null; + + OptimizedObjectInputStream objIn = null; + + try { + objIn = OptimizedObjectStreamRegistry.in(); + + objIn.context(clsMap, ctx, mapper, clsLdr != null ? clsLdr : dfltClsLdr); + + objIn.in().inputStream(in); + + return (T)objIn.readObject(); + } + catch (ClassNotFoundException e) { + throw new IgniteCheckedException("Failed to find class with given class loader for unmarshalling " + + "(make sure same versions of all classes are available on all nodes or enable peer-class-loading) " + + "[clsLdr=" + clsLdr + ", cls=" + e.getMessage() + "]", e); + } + catch (Exception e) { + throw new IgniteCheckedException("Failed to deserialize object with given class loader: " + clsLdr, e); + } + finally { + OptimizedObjectStreamRegistry.closeIn(objIn); + } + } + + /** {@inheritDoc} */ + @SuppressWarnings("unchecked") + @Override protected T unmarshal0(byte[] arr, @Nullable ClassLoader clsLdr) throws IgniteCheckedException { + assert arr != null; + + OptimizedObjectInputStream objIn = null; + + try { + objIn = OptimizedObjectStreamRegistry.in(); + + objIn.context(clsMap, ctx, mapper, clsLdr != null ? clsLdr : dfltClsLdr); + + objIn.in().bytes(arr, arr.length); + + return (T)objIn.readObject(); + } + catch (ClassNotFoundException e) { + throw new IgniteCheckedException("Failed to find class with given class loader for unmarshalling " + + "(make sure same version of all classes are available on all nodes or enable peer-class-loading)" + + " [clsLdr=" + clsLdr + ", cls=" + e.getMessage() + "]", e); + } + catch (Exception e) { + throw new IgniteCheckedException("Failed to deserialize object with given class loader: " + clsLdr, e); + } + finally { + OptimizedObjectStreamRegistry.closeIn(objIn); + } + } + + /** + * Checks whether {@code GridOptimizedMarshaller} is able to work on the current JVM. + *

+ * As long as {@code GridOptimizedMarshaller} uses JVM-private API, which is not guaranteed + * to be available on all JVM, this method should be called to ensure marshaller could work properly. + *

+ * Result of this method is automatically checked in constructor. + * + * @return {@code true} if {@code GridOptimizedMarshaller} can work on the current JVM or + * {@code false} if it can't. + */ + @SuppressWarnings({"TypeParameterExtendsFinalClass", "ErrorNotRethrown"}) + public static boolean available() { + try { + Class unsafeCls = Unsafe.class; + + unsafeCls.getMethod("allocateInstance", Class.class); + unsafeCls.getMethod("copyMemory", Object.class, long.class, Object.class, long.class, long.class); + + return true; + } + catch (Exception ignored) { + return false; + } + catch (NoClassDefFoundError ignored) { + return false; + } + } + + /** + * Undeployment callback invoked when class loader is being undeployed. + * + * @param ldr Class loader being undeployed. + */ + @Override public void onUndeploy(ClassLoader ldr) { + for (Class cls : clsMap.keySet()) { + if (ldr.equals(cls.getClassLoader())) + clsMap.remove(cls); + } + + U.clearClassCache(ldr); + } +} http://git-wip-us.apache.org/repos/asf/ignite/blob/a1b5b8c3/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerIdMapper.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerIdMapper.java b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerIdMapper.java new file mode 100644 index 0000000..381e5d0 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/marshaller/optimized/OptimizedMarshallerIdMapper.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.marshaller.optimized; + +/** + * ID mapper. + */ +public interface OptimizedMarshallerIdMapper { + /** + * Gets type ID for provided class name. + *

+ * If {@code 0} is returned, hash code of class name will be used. + * + * @param clsName Class name. + * @return Type ID. + */ + public int typeId(String clsName); +} \ No newline at end of file