hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Weng (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4661) Add HTTPS for WebUIs on Branch-1
Date Tue, 06 Aug 2013 00:01:48 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Michael Weng updated MAPREDUCE-4661:

    Attachment: branch-1.2-patch.txt6

Fixed tasklog url and SN for HttpServer on running as daemon. Following is the change compared
to the previous patch.
diff --git a/src/core/org/apache/hadoop/http/HttpServer.java b/src/core/org/apache/hadoop/http/HttpServer.ja
index 0047d64..efcaad6 100644
--- a/src/core/org/apache/hadoop/http/HttpServer.java
+++ b/src/core/org/apache/hadoop/http/HttpServer.java
@@ -167,7 +167,6 @@ public class HttpServer implements FilterContainer {
     // default value (currently 250).
     QueuedThreadPool threadPool = maxThreads == -1 ?
         new QueuedThreadPool() : new QueuedThreadPool(maxThreads);
-    threadPool.setDaemon(true);
     final String appDir = getWebAppsPath();
diff --git a/src/mapred/org/apache/hadoop/mapred/JobHistory.java b/src/mapred/org/apache/hadoop/mapred/JobHi
index 4ba2e38..9d701f5 100644
--- a/src/mapred/org/apache/hadoop/mapred/JobHistory.java
+++ b/src/mapred/org/apache/hadoop/mapred/JobHistory.java
@@ -2787,7 +2787,7 @@ public class JobHistory {
    *         task-attempt-id are unavailable.
   public static String getTaskLogsUrl(JobHistory.TaskAttempt attempt) {
-    if (attempt.get(Keys.SHUFFLE_PORT).equals("")
+    if (attempt.get(Keys.HTTP_PORT).equals("")
         || attempt.get(Keys.TRACKER_NAME).equals("")
         || attempt.get(Keys.TASK_ATTEMPT_ID).equals("")) {
       return null;
@@ -2797,6 +2797,6 @@ public class JobHistory {
     return TaskLogServlet.getTaskLogUrl(taskTrackerName, attempt
-        .get(Keys.SHUFFLE_PORT), attempt.get(Keys.TASK_ATTEMPT_ID));
+        .get(Keys.HTTP_PORT), attempt.get(Keys.TASK_ATTEMPT_ID));

Also attached the new patch.
> Add HTTPS for WebUIs on Branch-1
> --------------------------------
>                 Key: MAPREDUCE-4661
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4661
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: security, webapps
>    Affects Versions: 1.0.3
>            Reporter: Plamen Jeliazkov
>            Assignee: Michael Weng
>         Attachments: branch-1.2-patch.txt, branch-1.2-patch.txt2, branch-1.2-patch.txt3,
branch-1.2-patch.txt4, branch-1.2-patch.txt5, branch-1.2-patch.txt6, MAPREDUCE-4461.patch,
MAPREDUCE-4661.patch, MAPREDUCE-4661.patch, MAPREDUCE-4661.patch
> After investigating the methodology used to add HTTPS support in branch-2, I feel that
this same approach should be back-ported to branch-1. I have taken many of the patches used
for branch-2 and merged them in.
> I was working on top of HDP 1 at the time - I will provide a patch for trunk soon once
I can confirm I am adding only the necessities for supporting HTTPS on the webUIs.
> As an added benefit -- this patch actually provides HTTPS webUI to HBase by extension.
If you take a hadoop-core jar compiled with this patch and put it into the hbase/lib directory
and apply the necessary configs to hbase/conf.
> ========= OLD IDEA(s) BEHIND ADDING HTTPS (look @ Sept 17th patch) ==========
> In order to provide full security around the cluster, the webUI should also be secure
if desired to prevent cookie theft and user masquerading. 
> Here is my proposed work. Currently I can only add HTTPS support. I do not know how to
switch reliance of the HttpServer from HTTP to HTTPS fully.
> In order to facilitate this change I propose the following configuration additions:
> mapred.https.enable -> false
> mapred.https.need.client.auth -> false
> mapred.https.server.keystore.resource -> "ssl-server.xml"
> mapred.job.tracker.https.port -> 50035
> mapred.job.tracker.https.address -> "<IP_ADDR>:50035"
> mapred.task.tracker.https.port -> 50065
> mapred.task.tracker.https.address -> "<IP_ADDR>:50065"
> I tested this on my local box after using keytool to generate a SSL certficate. You will
need to change ssl-server.xml to point to the .keystore file after. Truststore may not be
necessary; you can just point it to the keystore.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message