Return-Path: X-Original-To: apmail-ignite-commits-archive@minotaur.apache.org Delivered-To: apmail-ignite-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 755851812C for ; Thu, 8 Oct 2015 04:15:38 +0000 (UTC) Received: (qmail 9988 invoked by uid 500); 8 Oct 2015 04:15:38 -0000 Delivered-To: apmail-ignite-commits-archive@ignite.apache.org Received: (qmail 9911 invoked by uid 500); 8 Oct 2015 04:15:38 -0000 Mailing-List: contact commits-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list commits@ignite.apache.org Received: (qmail 7765 invoked by uid 99); 8 Oct 2015 04:15:36 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Oct 2015 04:15:36 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 5D3EDE0BAF; Thu, 8 Oct 2015 04:15:36 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: anovikov@apache.org To: commits@ignite.apache.org Date: Thu, 08 Oct 2015 04:16:23 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [49/50] ignite git commit: Merge ignite-1168 into ignite-843 Merge ignite-1168 into ignite-843 Project: http://git-wip-us.apache.org/repos/asf/ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/1b933c6b Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/1b933c6b Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/1b933c6b Branch: refs/heads/ignite-843 Commit: 1b933c6b1672355c4e79f784aa93cea4330f2ade Parents: 7693eed 5ef7c38 Author: Andrey Authored: Thu Oct 8 11:14:58 2015 +0700 Committer: Andrey Committed: Thu Oct 8 11:14:58 2015 +0700 ---------------------------------------------------------------------- RELEASE_NOTES.txt | 2 + assembly/dependencies-fabric.xml | 2 +- assembly/release-fabric.xml | 10 +- bin/igniterouter.bat | 2 +- bin/igniterouter.sh | 2 +- examples/schema-import/pom-standalone.xml | 90 ++++ examples/schema-import/pom.xml | 23 +- .../computegrid/ComputeClosureExample.java | 2 +- modules/apache-license-gen/README.txt | 33 ++ .../JettyRestProcessorAbstractSelfTest.java | 186 +++++++- .../TcpDiscoveryCloudIpFinderSelfTest.java | 2 + .../org/apache/ignite/IgniteAtomicLong.java | 2 +- .../org/apache/ignite/IgniteFileSystem.java | 2 + .../apache/ignite/IgniteSystemProperties.java | 3 + .../java/org/apache/ignite/igfs/IgfsPath.java | 2 +- .../apache/ignite/internal/IgniteKernal.java | 74 ++-- .../managers/discovery/CustomEventListener.java | 4 +- .../discovery/GridDiscoveryManager.java | 57 +-- .../cache/DynamicCacheChangeRequest.java | 19 + .../cache/DynamicCacheDescriptor.java | 46 +- .../processors/cache/GridCacheContext.java | 2 +- .../processors/cache/GridCacheEventManager.java | 12 +- .../cache/GridCacheExplicitLockSpan.java | 13 +- .../processors/cache/GridCacheIoManager.java | 76 +--- .../processors/cache/GridCacheMessage.java | 7 - .../cache/GridCacheMvccCandidate.java | 5 +- .../processors/cache/GridCacheMvccManager.java | 67 +-- .../GridCachePartitionExchangeManager.java | 98 ++--- .../processors/cache/GridCacheProcessor.java | 135 ++++-- .../cache/GridCacheSharedContext.java | 32 -- .../processors/cache/GridCacheUtils.java | 8 - .../processors/cache/IgniteCacheProxy.java | 4 +- .../distributed/GridDistributedCacheEntry.java | 2 +- .../dht/GridDhtAffinityAssignmentRequest.java | 5 - .../dht/GridDhtAffinityAssignmentResponse.java | 5 - .../cache/distributed/dht/GridDhtGetFuture.java | 4 +- .../distributed/dht/GridDhtLockRequest.java | 10 +- .../distributed/dht/GridDhtTxFinishRequest.java | 5 - .../distributed/dht/GridDhtTxPrepareFuture.java | 6 +- .../dht/GridDhtTxPrepareRequest.java | 7 +- .../dht/GridPartitionedGetFuture.java | 5 +- .../dht/atomic/GridDhtAtomicUpdateFuture.java | 5 + .../dht/atomic/GridNearAtomicUpdateFuture.java | 9 +- .../dht/colocated/GridDhtColocatedCache.java | 12 +- .../colocated/GridDhtColocatedLockFuture.java | 16 +- .../dht/preloader/GridDhtForceKeysRequest.java | 5 - .../dht/preloader/GridDhtForceKeysResponse.java | 5 - .../GridDhtPartitionDemandMessage.java | 5 - .../GridDhtPartitionSupplyMessage.java | 5 - .../GridDhtPartitionsAbstractMessage.java | 5 - .../dht/preloader/GridDhtPreloader.java | 8 +- .../distributed/near/GridNearGetFuture.java | 2 + .../near/GridNearTxFinishFuture.java | 4 +- .../cache/distributed/near/GridNearTxLocal.java | 21 +- .../cache/local/GridLocalCacheEntry.java | 2 +- .../continuous/CacheContinuousQueryHandler.java | 10 +- .../continuous/CacheContinuousQueryManager.java | 66 +-- .../transactions/IgniteTxLocalAdapter.java | 35 +- .../continuous/GridContinuousProcessor.java | 20 +- .../datastructures/DataStructuresProcessor.java | 6 +- .../internal/processors/igfs/IgfsFileInfo.java | 15 +- .../internal/processors/igfs/IgfsImpl.java | 121 +----- .../processors/igfs/IgfsMetaManager.java | 427 ++++++++++++++++--- .../processors/rest/GridRestProcessor.java | 354 +++++++++++++-- .../handlers/cache/GridCacheCommandHandler.java | 40 +- .../top/GridTopologyCommandHandler.java | 23 +- .../ignite/internal/util/GridArgumentCheck.java | 5 +- .../ignite/internal/util/IgniteUtils.java | 2 +- .../util/nio/GridNioRecoveryDescriptor.java | 4 +- .../org/apache/ignite/mxbean/IgniteMXBean.java | 8 +- .../communication/tcp/TcpCommunicationSpi.java | 7 +- .../discovery/DiscoverySpiCustomMessage.java | 12 +- .../ignite/spi/discovery/tcp/ServerImpl.java | 398 ++++++++++++----- .../spi/discovery/tcp/TcpDiscoveryImpl.java | 6 +- .../spi/discovery/tcp/TcpDiscoverySpi.java | 2 +- .../tcp/internal/TcpDiscoveryNodesRing.java | 94 ++-- .../messages/TcpDiscoveryDiscardMessage.java | 15 +- .../TcpDiscoveryNodeAddFinishedMessage.java | 2 +- .../messages/TcpDiscoveryNodeAddedMessage.java | 19 +- .../ignite/igfs/IgfsFragmentizerSelfTest.java | 2 +- .../internal/GridUpdateNotifierSelfTest.java | 1 + .../IgniteClientReconnectAbstractTest.java | 35 +- .../IgniteClientReconnectCacheTest.java | 154 +++++++ .../GridDiscoveryManagerAliveCacheSelfTest.java | 2 + .../processors/cache/CrossCacheLockTest.java | 142 ++++++ .../GridCacheAbstractFailoverSelfTest.java | 3 + .../GridCacheAbstractRemoveFailureTest.java | 9 +- .../GridCacheDeploymentOffHeapSelfTest.java | 15 + .../GridCacheFinishPartitionsSelfTest.java | 5 +- .../GridCacheVariableTopologySelfTest.java | 3 +- ...IgniteCacheAtomicPutAllFailoverSelfTest.java | 4 + .../cache/IgniteCacheCreateRestartSelfTest.java | 3 + .../cache/IgniteCacheEntryListenerTxTest.java | 4 + .../IgniteCacheP2pUnmarshallingErrorTest.java | 7 + ...CacheP2pUnmarshallingRebalanceErrorTest.java | 12 +- .../cache/IgniteCachePutAllRestartTest.java | 4 +- .../cache/IgniteDynamicCacheFilterTest.java | 150 +++++++ .../IgniteTxExceptionAbstractSelfTest.java | 3 + ...omicOffheapQueueCreateMultiNodeSelfTest.java | 5 - ...ionedAtomicQueueCreateMultiNodeSelfTest.java | 9 +- ...artitionedOffHeapValuesQueueApiSelfTest.java | 4 + ...PartitionedQueueCreateMultiNodeSelfTest.java | 16 +- ...nedQueueFailoverDataConsistencySelfTest.java | 5 - ...GridCachePartitionedSetFailoverSelfTest.java | 4 + .../distributed/CacheAffEarlySelfTest.java | 245 ----------- .../distributed/CacheAffinityEarlyTest.java | 168 ++++++++ ...acheAsyncOperationsFailoverAbstractTest.java | 11 + .../CacheGetFutureHangsSelfTest.java | 156 ++++--- .../CachePutAllFailoverAbstractTest.java | 11 + .../IgniteCacheAtomicNodeRestartTest.java | 8 + .../IgniteCacheCreatePutMultiNodeSelfTest.java | 151 +++++++ .../distributed/IgniteCacheCreatePutTest.java | 125 ++++++ .../IgniteCachePutGetRestartAbstractTest.java | 3 + .../IgniteCacheSizeFailoverTest.java | 3 + .../IgniteTxPreloadAbstractTest.java | 43 +- .../GridCacheColocatedTxExceptionSelfTest.java | 5 - .../dht/GridNearCacheTxNodeFailureSelfTest.java | 4 + ...gniteAtomicLongChangingTopologySelfTest.java | 29 +- .../IgniteCacheCrossCacheTxFailoverTest.java | 3 + .../IgniteCrossCacheTxNearEnabledSelfTest.java | 28 ++ .../dht/IgniteCrossCacheTxSelfTest.java | 213 +++++++++ ...GridCacheValueConsistencyAtomicSelfTest.java | 2 +- ...ledFairAffinityMultiNodeFullApiSelfTest.java | 4 + .../near/GridCacheNearTxExceptionSelfTest.java | 9 +- .../near/NearCacheMultithreadedUpdateTest.java | 217 ++++++++++ .../GridCacheReplicatedInvalidateSelfTest.java | 249 ----------- .../GridCacheReplicatedTxExceptionSelfTest.java | 5 - .../replicated/GridReplicatedTxPreloadTest.java | 2 - ...eCacheExpiryPolicyWithStoreAbstractTest.java | 5 +- .../GridCacheLocalTxExceptionSelfTest.java | 5 - ...bledFairAffinityMultiJvmFullApiSelfTest.java | 5 + ...tomicNearEnabledMultiJvmFullApiSelfTest.java | 5 + ...ontinuousQueryReplicatedOneNodeSelfTest.java | 120 ++++++ .../DataStreamerMultiThreadedSelfTest.java | 4 +- .../DataStreamerMultinodeCreateCacheTest.java | 2 + .../processors/igfs/IgfsAbstractSelfTest.java | 203 ++++++--- .../igfs/IgfsMetaManagerSelfTest.java | 6 - .../processors/igfs/IgfsMetricsSelfTest.java | 2 +- .../processors/igfs/IgfsProcessorSelfTest.java | 29 +- .../igfs/UniversalFileSystemAdapter.java | 1 - ...lientDiscoverySpiFailureTimeoutSelfTest.java | 33 +- .../tcp/TcpClientDiscoverySpiSelfTest.java | 2 + .../tcp/TcpDiscoveryMultiThreadedTest.java | 53 ++- .../spi/discovery/tcp/TcpDiscoverySelfTest.java | 315 +++++++++++++- .../multijvm/IgniteCacheProcessProxy.java | 3 +- .../IgniteCacheFailoverTestSuite.java | 4 - .../IgniteCacheFailoverTestSuite3.java | 28 +- .../testsuites/IgniteCacheTestSuite3.java | 2 - .../testsuites/IgniteCacheTestSuite4.java | 24 ++ .../processors/hadoop/igfs/HadoopIgfsUtils.java | 36 ++ .../ignite/igfs/Hadoop1DualAbstractTest.java | 5 + ...oopFileSystemUniversalFileSystemAdapter.java | 4 +- .../HadoopIgfs20FileSystemAbstractSelfTest.java | 7 +- .../IgniteHadoopFileSystemAbstractSelfTest.java | 5 +- .../processors/query/h2/sql/GridSqlArray.java | 52 +++ .../processors/query/h2/sql/GridSqlElement.java | 2 +- .../query/h2/sql/GridSqlFunction.java | 60 ++- .../query/h2/sql/GridSqlFunctionType.java | 3 + .../query/h2/sql/GridSqlPlaceholder.java | 7 +- .../query/h2/sql/GridSqlQueryParser.java | 84 ++-- .../processors/query/h2/sql/GridSqlType.java | 29 +- .../query/h2/sql/GridQueryParsingTest.java | 27 ++ .../IgniteCacheQuerySelfTestSuite.java | 4 +- .../http/jetty/GridJettyJsonConfig.java | 2 +- .../http/jetty/GridJettyRestHandler.java | 4 +- modules/yardstick/pom.xml | 10 +- pom.xml | 36 +- 167 files changed, 4566 insertions(+), 1823 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/1b933c6b/modules/clients/src/test/java/org/apache/ignite/internal/processors/rest/JettyRestProcessorAbstractSelfTest.java ---------------------------------------------------------------------- diff --cc modules/clients/src/test/java/org/apache/ignite/internal/processors/rest/JettyRestProcessorAbstractSelfTest.java index 8aecb61,bb6e67e..68659e8 --- a/modules/clients/src/test/java/org/apache/ignite/internal/processors/rest/JettyRestProcessorAbstractSelfTest.java +++ b/modules/clients/src/test/java/org/apache/ignite/internal/processors/rest/JettyRestProcessorAbstractSelfTest.java @@@ -26,7 -24,7 +26,8 @@@ import java.net.HttpURLConnection import java.net.URL; import java.net.URLConnection; import java.net.URLEncoder; + import java.util.Collection; +import java.nio.charset.Charset; import java.util.HashMap; import java.util.Iterator; import java.util.List; @@@ -34,18 -32,24 +35,26 @@@ import java.util.Map import java.util.UUID; import java.util.concurrent.ConcurrentHashMap; import java.util.regex.Pattern; + import net.sf.json.JSONNull; import net.sf.json.JSONObject; +import org.apache.ignite.Ignite; import org.apache.ignite.IgniteCache; import org.apache.ignite.cache.CachePeekMode; import org.apache.ignite.cache.query.SqlQuery; import org.apache.ignite.cache.query.annotations.QuerySqlField; import org.apache.ignite.cluster.ClusterNode; import org.apache.ignite.configuration.CacheConfiguration; + import org.apache.ignite.internal.processors.cache.GridCacheProcessor; + import org.apache.ignite.internal.processors.cache.IgniteCacheProxy; + import org.apache.ignite.internal.processors.cache.IgniteInternalCache; + import org.apache.ignite.internal.processors.cache.query.GridCacheSqlIndexMetadata; + import org.apache.ignite.internal.processors.cache.query.GridCacheSqlMetadata; +import org.apache.ignite.internal.processors.json.IgniteJsonString; import org.apache.ignite.internal.processors.rest.handlers.GridRestCommandHandler; import org.apache.ignite.internal.util.typedef.F; + import org.apache.ignite.internal.util.typedef.internal.CU; import org.apache.ignite.lang.IgniteBiPredicate; + import org.apache.ignite.lang.IgnitePredicate; import org.apache.ignite.testframework.GridTestUtils; import static org.apache.ignite.IgniteSystemProperties.IGNITE_JETTY_PORT; @@@ -1544,182 -1406,7 +1686,180 @@@ public abstract class JettyRestProcesso assertFalse(queryCursorFound()); } - - - protected abstract String signature() throws Exception; + /** + * @throws Exception If failed. + */ + public void testRunScriptPost() throws Exception { + String f = "function(param){return param;}"; + String ret = makePostRequest(F.asMap("cmd", GridRestCommand.RUN_SCRIPT.key(), "func", URLEncoder.encode(f)), "{\"arg\":\"hello\"}"); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + jsonEquals(ret, stringPattern("hello", true)); + } + + /** + * @throws Exception If failed. + */ + public void testRunScript() throws Exception { + String f = "function(param){return param;}"; + String ret = content(F.asMap("cmd", GridRestCommand.RUN_SCRIPT.key(), "func", + URLEncoder.encode(f), "arg", "hello")); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + jsonEquals(ret, stringPattern("hello", true)); + } + + /** + * @throws Exception If failed. + */ + public void testRunAffinityScriptPost() throws Exception { + ClusterNode node = grid(0).affinity(null).mapKeyToNode(new IgniteJsonString("key0")); + + Ignite ignite = null; + + for (int i = 0; i < GRID_CNT; ++i) { + if (grid(i).localNode().equals(node)) + ignite = grid(i); + } + + assertNotNull(ignite); + + String f = "function(expName){"+ + "if (expName !== \"hello\") {" + + "throw \"Not correct arg.\"" + + "}" + + "return ignite.name();}"; + + String ret = makePostRequest(F.asMap("cmd", GridRestCommand.AFFINITY_RUN_SCRIPT.key(), + "func", URLEncoder.encode(f)), + "{\"arg\":\"" + "hello" + "\",\"key\":\"key0\"}"); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + jsonEquals(ret, stringPattern(ignite.name(), true)); + } + + /** + * @throws Exception If failed. + */ + public void testRunAffinityScript() throws Exception { + ClusterNode node = grid(0).affinity(null).mapKeyToNode("key0"); + + Ignite ignite = null; + + for (int i = 0; i < GRID_CNT; ++i) { + if (grid(i).localNode().equals(node)) + ignite = grid(i); + } + + assertNotNull(ignite); + + String f = "function(expName){"+ + "if (expName !== \"hello\") {" + + "throw \"Not correct arg.\"" + + "}" + + "return ignite.name();}"; + + String ret = content(F.asMap("cmd", GridRestCommand.AFFINITY_RUN_SCRIPT.key(), + "func", URLEncoder.encode(f), "key", "key0", "arg", "hello")); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + jsonEquals(ret, stringPattern(ignite.name(), true)); + } + + /** + * @throws Exception If failed. + */ + public void testMapReduceScriptPost() throws Exception { + String map = "function(nodes, arg) {" + + "var words = arg.split(\" \");" + + "for (var i = 0; i < words.length; i++) {" + + "var f = function(word) {" + + "return word.length;" + + "};" + + "emit(f, words[i], nodes[i % nodes.length]);" + + "}"+ + "};"; + + String reduce = "function(results) {"+ + "var sum = 0;"+ + "for (var i = 0; i < results.size(); ++i) {"+ + "sum += results.get(i).intValue();"+ + "}" + + "return sum;" + + "};"; + + String ret = makePostRequest(F.asMap("cmd", GridRestCommand.EXECUTE_MAP_REDUCE_SCRIPT.key(), + "map", URLEncoder.encode(map), + "reduce", URLEncoder.encode(reduce)), "{\"arg\": \"Hello world!\"}"); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + jsonEquals(ret, integerPattern(11, true)); + } + + /** + * @throws Exception If failed. + */ + public void testMapReduceScript() throws Exception { + String map = "function(nodes, arg) {" + + "var words = arg.split(\" \");" + + "for (var i = 0; i < words.length; i++) {" + + "var f = function(word) {" + + "return word.length;" + + "};" + + "emit(f, words[i], nodes[i % nodes.length]);" + + "}"+ + "};"; + + String reduce = "function(results) {"+ + "var sum = 0;"+ + "for (var i = 0; i < results.size(); ++i) {"+ + "sum += results.get(i).intValue();"+ + "}" + + "return sum;" + + "};"; + + String ret = content(F.asMap("cmd", GridRestCommand.EXECUTE_MAP_REDUCE_SCRIPT.key(), + "map", URLEncoder.encode(map), + "reduce", URLEncoder.encode(reduce), + "arg", URLEncoder.encode("Hello world!"))); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + jsonEquals(ret, integerPattern(11, true)); + } + + /** + * @throws Exception If failed. + */ + public void testQueryArgsPost() throws Exception { + String qry = "salary > ? and salary <= ?"; + + String ret = makePostRequest(F.asMap("cmd", GridRestCommand.EXECUTE_SQL_QUERY.key(), + "type", "Person", "pageSize", "10", "cacheName", "person", + "qry", URLEncoder.encode(qry)), "{\"arg\": [1000, 2000]}"); + + assertNotNull(ret); + assertTrue(!ret.isEmpty()); + + JSONObject json = JSONObject.fromObject(ret); + + List items = (List)((Map)json.get("response")).get("items"); + + assertEquals(2, items.size()); + + assertFalse(queryCursorFound()); + } /** * @return True if any query cursor is available. http://git-wip-us.apache.org/repos/asf/ignite/blob/1b933c6b/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/1b933c6b/modules/core/src/main/java/org/apache/ignite/internal/processors/rest/GridRestProcessor.java ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/1b933c6b/modules/rest-http/src/main/java/org/apache/ignite/internal/processors/rest/protocols/http/jetty/GridJettyRestHandler.java ---------------------------------------------------------------------- diff --cc modules/rest-http/src/main/java/org/apache/ignite/internal/processors/rest/protocols/http/jetty/GridJettyRestHandler.java index 3684491,55bd842..9a89f43 --- a/modules/rest-http/src/main/java/org/apache/ignite/internal/processors/rest/protocols/http/jetty/GridJettyRestHandler.java +++ b/modules/rest-http/src/main/java/org/apache/ignite/internal/processors/rest/protocols/http/jetty/GridJettyRestHandler.java @@@ -673,15 -501,9 +673,15 @@@ public class GridJettyRestHandler exten case EXECUTE_SQL_FIELDS_QUERY: { RestQueryRequest restReq0 = new RestQueryRequest(); - restReq0.sqlQuery((String) params.get("qry")); + restReq0.sqlQuery((String)params.get("qry")); - restReq0.arguments(values("arg", params).toArray()); + if (req.getHeader("Content-Type") != null && req.getHeader("Content-Type").contains("json")) { + Map o = parseRequest(req); + List args = (List) ctx.scripting().toScriptObject(o.get("arg")); + restReq0.arguments(args.toArray()); + } + else + restReq0.arguments(values("arg", params).toArray()); restReq0.typeName((String) params.get("type")); http://git-wip-us.apache.org/repos/asf/ignite/blob/1b933c6b/pom.xml ----------------------------------------------------------------------