Return-Path: X-Original-To: apmail-zeppelin-commits-archive@minotaur.apache.org Delivered-To: apmail-zeppelin-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7FBE719D1E for ; Thu, 28 Apr 2016 04:00:55 +0000 (UTC) Received: (qmail 15728 invoked by uid 500); 28 Apr 2016 04:00:55 -0000 Delivered-To: apmail-zeppelin-commits-archive@zeppelin.apache.org Received: (qmail 15692 invoked by uid 500); 28 Apr 2016 04:00:55 -0000 Mailing-List: contact commits-help@zeppelin.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zeppelin.incubator.apache.org Delivered-To: mailing list commits@zeppelin.incubator.apache.org Received: (qmail 15683 invoked by uid 99); 28 Apr 2016 04:00:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2016 04:00:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D4B76180233 for ; Thu, 28 Apr 2016 04:00:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.221 X-Spam-Level: X-Spam-Status: No, score=-3.221 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Gk1G-oTj1TYp for ; Thu, 28 Apr 2016 04:00:50 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 444AA5F366 for ; Thu, 28 Apr 2016 04:00:49 +0000 (UTC) Received: (qmail 14737 invoked by uid 99); 28 Apr 2016 04:00:48 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2016 04:00:48 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 392EFDFB8F; Thu, 28 Apr 2016 04:00:48 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: felixcheung@apache.org To: commits@zeppelin.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-zeppelin git commit: [ZEPPELIN-798] Migrate to Jetty version 9 Date: Thu, 28 Apr 2016 04:00:48 +0000 (UTC) Repository: incubator-zeppelin Updated Branches: refs/heads/master ec661d7c1 -> bd4eaf8a3 [ZEPPELIN-798] Migrate to Jetty version 9 ### What is this PR for? Upgrade from Jetty 8 to Jetty 9 mainly for deadlock bug fix https://bugs.eclipse.org/bugs/show_bug.cgi?id=389645 Jetty 9 has changed the maven names and context building (for web and websocket). ### What type of PR is it? [Bug Fix] ### Todos * [x ] - Adapt the unit tests ### What is the Jira issue? * Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN-798 ### How should this be tested? Outline the steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Eric Charles Closes #831 from echarles/jetty9 and squashes the following commits: 840bdb4 [Eric Charles] Use correct syntax for httpConfig and httpsConfig, thx @jongyoul for the review 8e3daeb [Eric Charles] Merge branch 'master' into jetty9 05d9c50 [Eric Charles] Use jetty 9.2.15.v20160210 which is compatible with JDK-1.7 2a111d8 [Eric Charles] Exclude jetty websocket-client from htmlunit to ensure dependeny convergence c48d4b2 [Eric Charles] Add javadoc to NotebookWebSocketCreator class 82d311a [Eric Charles] Upgrade to jetty 9 Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/bd4eaf8a Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/bd4eaf8a Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/bd4eaf8a Branch: refs/heads/master Commit: bd4eaf8a35370f5f702444e576058b8c1c920acd Parents: ec661d7 Author: Eric Charles Authored: Thu Apr 21 08:53:43 2016 +0200 Committer: Felix Cheung Committed: Wed Apr 27 21:00:44 2016 -0700 ---------------------------------------------------------------------- zeppelin-server/pom.xml | 19 ++- .../apache/zeppelin/server/ZeppelinServer.java | 116 ++++++++++--------- .../apache/zeppelin/socket/NotebookServer.java | 15 +-- .../apache/zeppelin/socket/NotebookSocket.java | 21 ++-- .../zeppelin/socket/NotebookSocketListener.java | 6 +- .../socket/NotebookWebSocketCreator.java | 36 ++++++ 6 files changed, 132 insertions(+), 81 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/bd4eaf8a/zeppelin-server/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-server/pom.xml b/zeppelin-server/pom.xml index f2c9ced..0d9da0b 100644 --- a/zeppelin-server/pom.xml +++ b/zeppelin-server/pom.xml @@ -35,7 +35,7 @@ 2.7.7 - 8.1.14.v20131031 + 9.2.15.v20160210 4.3.6 @@ -129,11 +129,16 @@ - org.eclipse.jetty.aggregate - jetty-all-server + org.eclipse.jetty + jetty-webapp ${jetty.version} - + + org.eclipse.jetty.websocket + websocket-server + ${jetty.version} + + com.google.code.gson gson @@ -237,6 +242,10 @@ xml-apis xml-apis + + org.eclipse.jetty.websocket + websocket-client + http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/bd4eaf8a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java index d57aa71..13be8b3 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java @@ -31,13 +31,10 @@ import org.apache.zeppelin.scheduler.SchedulerFactory; import org.apache.zeppelin.search.LuceneSearch; import org.apache.zeppelin.search.SearchService; import org.apache.zeppelin.socket.NotebookServer; -import org.eclipse.jetty.server.AbstractConnector; -import org.eclipse.jetty.server.Handler; -import org.eclipse.jetty.server.Server; +import org.eclipse.jetty.http.HttpVersion; +import org.eclipse.jetty.server.*; import org.eclipse.jetty.server.handler.ContextHandlerCollection; -import org.eclipse.jetty.server.nio.SelectChannelConnector; import org.eclipse.jetty.server.session.SessionHandler; -import org.eclipse.jetty.server.ssl.SslSelectChannelConnector; import org.eclipse.jetty.servlet.DefaultServlet; import org.eclipse.jetty.servlet.FilterHolder; import org.eclipse.jetty.servlet.ServletContextHandler; @@ -47,7 +44,6 @@ import org.eclipse.jetty.webapp.WebAppContext; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import javax.net.ssl.SSLContext; import javax.servlet.DispatcherType; import javax.ws.rs.core.Application; import java.io.File; @@ -58,7 +54,6 @@ import java.util.Set; /** * Main class of Zeppelin. - * */ public class ZeppelinServer extends Application { private static final Logger LOG = LoggerFactory.getLogger(ZeppelinServer.class); @@ -91,24 +86,26 @@ public class ZeppelinServer extends Application { } public static void main(String[] args) throws InterruptedException { + ZeppelinConfiguration conf = ZeppelinConfiguration.create(); conf.setProperty("args", args); - // REST api - final ServletContextHandler restApiContext = setupRestApiContextHandler(conf); + jettyWebServer = setupJettyServer(conf); - // Notebook server - final ServletContextHandler notebookContext = setupNotebookServer(conf); + ContextHandlerCollection contexts = new ContextHandlerCollection(); + jettyWebServer.setHandler(contexts); // Web UI - final WebAppContext webApp = setupWebAppContext(conf); + final WebAppContext webApp = setupWebAppContext(contexts, conf); - // add all handlers - ContextHandlerCollection contexts = new ContextHandlerCollection(); - contexts.setHandlers(new Handler[]{restApiContext, notebookContext, webApp}); + // REST api + setupRestApiContextHandler(webApp, conf); - jettyWebServer = setupJettyServer(conf); - jettyWebServer.setHandler(contexts); + // Notebook server + setupNotebookServer(webApp, conf); + + //Below is commented since zeppelin-docs module is removed. + //final WebAppContext webAppSwagg = setupWebAppSwagger(conf); LOG.info("Starting zeppelin server"); try { @@ -150,27 +147,50 @@ public class ZeppelinServer extends Application { } private static Server setupJettyServer(ZeppelinConfiguration conf) { - AbstractConnector connector; + + final Server server = new Server(); + ServerConnector connector; + if (conf.useSsl()) { - connector = new SslSelectChannelConnector(getSslContextFactory(conf)); + + HttpConfiguration httpConfig = new HttpConfiguration(); + httpConfig.setSecureScheme("https"); + httpConfig.setSecurePort(conf.getServerPort()); + httpConfig.setOutputBufferSize(32768); + + HttpConfiguration httpsConfig = new HttpConfiguration(httpConfig); + SecureRequestCustomizer src = new SecureRequestCustomizer(); + // Only with Jetty 9.3.x +// src.setStsMaxAge(2000); +// src.setStsIncludeSubDomains(true); + httpsConfig.addCustomizer(src); + + connector = new ServerConnector( + server, + new SslConnectionFactory(getSslContextFactory(conf), HttpVersion.HTTP_1_1.asString()), + new HttpConnectionFactory(httpsConfig)); + + } else { - connector = new SelectChannelConnector(); + + connector = new ServerConnector(server); + } // Set some timeout options to make debugging easier. int timeout = 1000 * 30; - connector.setMaxIdleTime(timeout); + connector.setIdleTimeout(timeout); connector.setSoLingerTime(-1); connector.setHost(conf.getServerAddress()); connector.setPort(conf.getServerPort()); - final Server server = new Server(); server.addConnector(connector); return server; } - private static ServletContextHandler setupNotebookServer(ZeppelinConfiguration conf) { + private static void setupNotebookServer(WebAppContext webapp, + ZeppelinConfiguration conf) { notebookWsServer = new NotebookServer(); String maxTextMessageSize = conf.getWebsocketMaxTextMessageSize(); final ServletHolder servletHolder = new ServletHolder(notebookWsServer); @@ -179,28 +199,23 @@ public class ZeppelinServer extends Application { final ServletContextHandler cxfContext = new ServletContextHandler( ServletContextHandler.SESSIONS); - cxfContext.setSessionHandler(new SessionHandler()); - cxfContext.setContextPath(conf.getServerContextPath()); - cxfContext.addServlet(servletHolder, "/ws/*"); - cxfContext.addFilter(new FilterHolder(CorsFilter.class), "/*", - EnumSet.allOf(DispatcherType.class)); - return cxfContext; + webapp.addServlet(servletHolder, "/ws/*"); + webapp.addFilter(new FilterHolder(CorsFilter.class), "/*", + EnumSet.allOf(DispatcherType.class)); + } - @SuppressWarnings("deprecation") private static SslContextFactory getSslContextFactory(ZeppelinConfiguration conf) { - // Note that the API for the SslContextFactory is different for - // Jetty version 9 SslContextFactory sslContextFactory = new SslContextFactory(); // Set keystore - sslContextFactory.setKeyStore(conf.getKeyStorePath()); + sslContextFactory.setKeyStorePath(conf.getKeyStorePath()); sslContextFactory.setKeyStoreType(conf.getKeyStoreType()); sslContextFactory.setKeyStorePassword(conf.getKeyStorePassword()); sslContextFactory.setKeyManagerPassword(conf.getKeyManagerPassword()); // Set truststore - sslContextFactory.setTrustStore(conf.getTrustStorePath()); + sslContextFactory.setTrustStorePath(conf.getTrustStorePath()); sslContextFactory.setTrustStoreType(conf.getTrustStoreType()); sslContextFactory.setTrustStorePassword(conf.getTrustStorePassword()); @@ -209,43 +224,31 @@ public class ZeppelinServer extends Application { return sslContextFactory; } - @SuppressWarnings("unused") //TODO(bzz) why unused? - private static SSLContext getSslContext(ZeppelinConfiguration conf) - throws Exception { + private static void setupRestApiContextHandler(WebAppContext webapp, + ZeppelinConfiguration conf) { - SslContextFactory scf = getSslContextFactory(conf); - if (!scf.isStarted()) { - scf.start(); - } - return scf.getSslContext(); - } - - private static ServletContextHandler setupRestApiContextHandler(ZeppelinConfiguration conf) { final ServletHolder cxfServletHolder = new ServletHolder(new CXFNonSpringJaxrsServlet()); cxfServletHolder.setInitParameter("javax.ws.rs.Application", ZeppelinServer.class.getName()); cxfServletHolder.setName("rest"); cxfServletHolder.setForcedPath("rest"); - final ServletContextHandler cxfContext = new ServletContextHandler(); - cxfContext.setSessionHandler(new SessionHandler()); - cxfContext.setContextPath(conf.getServerContextPath()); - cxfContext.addServlet(cxfServletHolder, "/api/*"); + webapp.setSessionHandler(new SessionHandler()); + webapp.addServlet(cxfServletHolder, "/api/*"); - cxfContext.addFilter(new FilterHolder(CorsFilter.class), "/*", + webapp.addFilter(new FilterHolder(CorsFilter.class), "/*", EnumSet.allOf(DispatcherType.class)); - cxfContext.setInitParameter("shiroConfigLocations", + webapp.setInitParameter("shiroConfigLocations", new File(conf.getShiroPath()).toURI().toString()); - cxfContext.addFilter(org.apache.shiro.web.servlet.ShiroFilter.class, "/*", + webapp.addFilter(org.apache.shiro.web.servlet.ShiroFilter.class, "/*", EnumSet.allOf(DispatcherType.class)); - cxfContext.addEventListener(new org.apache.shiro.web.env.EnvironmentLoaderListener()); + webapp.addEventListener(new org.apache.shiro.web.env.EnvironmentLoaderListener()); - return cxfContext; } - private static WebAppContext setupWebAppContext( + private static WebAppContext setupWebAppContext(ContextHandlerCollection contexts, ZeppelinConfiguration conf) { WebAppContext webApp = new WebAppContext(); @@ -266,7 +269,10 @@ public class ZeppelinServer extends Application { } // Explicit bind to root webApp.addServlet(new ServletHolder(new DefaultServlet()), "/*"); + contexts.addHandler(webApp); + return webApp; + } @Override http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/bd4eaf8a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java index 9412d71..6ad3537 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java @@ -16,7 +16,6 @@ */ package org.apache.zeppelin.socket; - import java.io.IOException; import java.net.URISyntaxException; import java.net.UnknownHostException; @@ -48,15 +47,14 @@ import org.apache.zeppelin.server.ZeppelinServer; import org.apache.zeppelin.socket.Message.OP; import org.apache.zeppelin.ticket.TicketContainer; import org.apache.zeppelin.utils.SecurityUtils; -import org.eclipse.jetty.websocket.WebSocket; -import org.eclipse.jetty.websocket.WebSocketServlet; +import org.eclipse.jetty.websocket.servlet.WebSocketServlet; +import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory; import org.quartz.SchedulerException; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * Zeppelin websocket service. - * */ public class NotebookServer extends WebSocketServlet implements NotebookSocketListener, JobListenerFactory, AngularObjectRegistryListener, @@ -71,6 +69,10 @@ public class NotebookServer extends WebSocketServlet implements } @Override + public void configure(WebSocketServletFactory factory) { + factory.setCreator(new NotebookWebSocketCreator(this)); + } + public boolean checkOrigin(HttpServletRequest request, String origin) { try { return SecurityUtils.isValidOrigin(origin, ZeppelinConfiguration.create()); @@ -82,8 +84,7 @@ public class NotebookServer extends WebSocketServlet implements return false; } - @Override - public WebSocket doWebSocketConnect(HttpServletRequest req, String protocol) { + public NotebookSocket doWebSocketConnect(HttpServletRequest req, String protocol) { return new NotebookSocket(req, protocol, this); } @@ -448,7 +449,7 @@ public class NotebookServer extends WebSocketServlet implements } } - private void updateNote(WebSocket conn, HashSet userAndRoles, + private void updateNote(NotebookSocket conn, HashSet userAndRoles, Notebook notebook, Message fromMessage) throws SchedulerException, IOException { String noteId = (String) fromMessage.get("id"); http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/bd4eaf8a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java index aceea45..5d68bf5 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java @@ -20,19 +20,19 @@ import java.io.IOException; import javax.servlet.http.HttpServletRequest; -import org.eclipse.jetty.websocket.WebSocket; +import org.eclipse.jetty.websocket.api.Session; +import org.eclipse.jetty.websocket.api.WebSocketAdapter; /** * Notebook websocket */ -public class NotebookSocket implements WebSocket.OnTextMessage{ +public class NotebookSocket extends WebSocketAdapter { - private Connection connection; + private Session connection; private NotebookSocketListener listener; private HttpServletRequest request; private String protocol; - public NotebookSocket(HttpServletRequest req, String protocol, NotebookSocketListener listener) { this.listener = listener; @@ -41,22 +41,22 @@ public class NotebookSocket implements WebSocket.OnTextMessage{ } @Override - public void onClose(int closeCode, String message) { + public void onWebSocketClose(int closeCode, String message) { listener.onClose(this, closeCode, message); } @Override - public void onOpen(Connection connection) { + public void onWebSocketConnect(Session connection) { this.connection = connection; listener.onOpen(this); } @Override - public void onMessage(String message) { + public void onWebSocketText(String message) { listener.onMessage(this, message); } - - + + public HttpServletRequest getRequest() { return request; } @@ -66,8 +66,7 @@ public class NotebookSocket implements WebSocket.OnTextMessage{ } public void send(String serializeMessage) throws IOException { - connection.sendMessage(serializeMessage); + connection.getRemote().sendString(serializeMessage); } - } http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/bd4eaf8a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocketListener.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocketListener.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocketListener.java index 77fed6e..6fc0734 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocketListener.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocketListener.java @@ -20,7 +20,7 @@ package org.apache.zeppelin.socket; * NoteboookSocket listener */ public interface NotebookSocketListener { - public void onClose(NotebookSocket socket, int code, String message); - public void onOpen(NotebookSocket socket); - public void onMessage(NotebookSocket socket, String message); + void onClose(NotebookSocket socket, int code, String message); + void onOpen(NotebookSocket socket); + void onMessage(NotebookSocket socket, String message); } http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/bd4eaf8a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookWebSocketCreator.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookWebSocketCreator.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookWebSocketCreator.java new file mode 100644 index 0000000..1b8e2f4 --- /dev/null +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookWebSocketCreator.java @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.socket; + +import org.eclipse.jetty.websocket.servlet.ServletUpgradeRequest; +import org.eclipse.jetty.websocket.servlet.ServletUpgradeResponse; +import org.eclipse.jetty.websocket.servlet.WebSocketCreator; + +/** + * Responsible to create the WebSockets for the NotebookServer. + */ +public class NotebookWebSocketCreator implements WebSocketCreator { + private NotebookServer notebookServer; + + public NotebookWebSocketCreator(NotebookServer notebookServer) { + this.notebookServer = notebookServer; + } + public Object createWebSocket(ServletUpgradeRequest request, ServletUpgradeResponse response) { + return new NotebookSocket(request.getHttpServletRequest(), "", notebookServer); + } + +}