Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 6 Aug 2013 00:01:48 +0000 (UTC)
From: "Michael Weng (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12607972.1347921552144.5199.1375747308658@arcas>
In-Reply-To: <JIRA.12607972.1347921552144@arcas>
References: <JIRA.12607972.1347921552144@arcas>
Subject: [jira] [Updated] (MAPREDUCE-4661) Add HTTPS for WebUIs on Branch-1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Weng updated MAPREDUCE-4661:
------------------------------------

    Attachment: branch-1.2-patch.txt6

Fixed tasklog url and SN for HttpServer on running as daemon. Following is the change compared to the previous patch.
-----------
diff --git a/src/core/org/apache/hadoop/http/HttpServer.java b/src/core/org/apache/hadoop/http/HttpServer.ja
index 0047d64..efcaad6 100644
--- a/src/core/org/apache/hadoop/http/HttpServer.java
+++ b/src/core/org/apache/hadoop/http/HttpServer.java
@@ -167,7 +167,6 @@ public class HttpServer implements FilterContainer {
     // default value (currently 250).
     QueuedThreadPool threadPool = maxThreads == -1 ?
         new QueuedThreadPool() : new QueuedThreadPool(maxThreads);
-    threadPool.setDaemon(true);
     webServer.setThreadPool(threadPool);
 
     final String appDir = getWebAppsPath();
diff --git a/src/mapred/org/apache/hadoop/mapred/JobHistory.java b/src/mapred/org/apache/hadoop/mapred/JobHi
index 4ba2e38..9d701f5 100644
--- a/src/mapred/org/apache/hadoop/mapred/JobHistory.java
+++ b/src/mapred/org/apache/hadoop/mapred/JobHistory.java
@@ -2787,7 +2787,7 @@ public class JobHistory {
    *         task-attempt-id are unavailable.
    */
   public static String getTaskLogsUrl(JobHistory.TaskAttempt attempt) {
-    if (attempt.get(Keys.SHUFFLE_PORT).equals("")
+    if (attempt.get(Keys.HTTP_PORT).equals("")
         || attempt.get(Keys.TRACKER_NAME).equals("")
         || attempt.get(Keys.TASK_ATTEMPT_ID).equals("")) {
       return null;
@@ -2797,6 +2797,6 @@ public class JobHistory {
       JobInProgress.convertTrackerNameToHostName(
         attempt.get(Keys.TRACKER_NAME));
     return TaskLogServlet.getTaskLogUrl(taskTrackerName, attempt
-        .get(Keys.SHUFFLE_PORT), attempt.get(Keys.TASK_ATTEMPT_ID));
+        .get(Keys.HTTP_PORT), attempt.get(Keys.TASK_ATTEMPT_ID));
   }
 }
-----------

Also attached the new patch.
                
> Add HTTPS for WebUIs on Branch-1
> --------------------------------
>
>                 Key: MAPREDUCE-4661
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4661
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: security, webapps
>    Affects Versions: 1.0.3
>            Reporter: Plamen Jeliazkov
>            Assignee: Michael Weng
>         Attachments: branch-1.2-patch.txt, branch-1.2-patch.txt2, branch-1.2-patch.txt3, branch-1.2-patch.txt4, branch-1.2-patch.txt5, branch-1.2-patch.txt6, MAPREDUCE-4461.patch, MAPREDUCE-4661.patch, MAPREDUCE-4661.patch, MAPREDUCE-4661.patch
>
>
> After investigating the methodology used to add HTTPS support in branch-2, I feel that this same approach should be back-ported to branch-1. I have taken many of the patches used for branch-2 and merged them in.
> I was working on top of HDP 1 at the time - I will provide a patch for trunk soon once I can confirm I am adding only the necessities for supporting HTTPS on the webUIs.
> As an added benefit -- this patch actually provides HTTPS webUI to HBase by extension. If you take a hadoop-core jar compiled with this patch and put it into the hbase/lib directory and apply the necessary configs to hbase/conf.
> ========= OLD IDEA(s) BEHIND ADDING HTTPS (look @ Sept 17th patch) ==========
> In order to provide full security around the cluster, the webUI should also be secure if desired to prevent cookie theft and user masquerading. 
> Here is my proposed work. Currently I can only add HTTPS support. I do not know how to switch reliance of the HttpServer from HTTP to HTTPS fully.
> In order to facilitate this change I propose the following configuration additions:
> CONFIG PROPERTY -> DEFAULT VALUE
> mapred.https.enable -> false
> mapred.https.need.client.auth -> false
> mapred.https.server.keystore.resource -> "ssl-server.xml"
> mapred.job.tracker.https.port -> 50035
> mapred.job.tracker.https.address -> "<IP_ADDR>:50035"
> mapred.task.tracker.https.port -> 50065
> mapred.task.tracker.https.address -> "<IP_ADDR>:50065"
> I tested this on my local box after using keytool to generate a SSL certficate. You will need to change ssl-server.xml to point to the .keystore file after. Truststore may not be necessary; you can just point it to the keystore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira