Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 25 Oct 2017 17:54:06 +0000 (UTC)
From: "Istvan Fajth (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13111993.1508943103000.59217.1508954046890@Atlassian.JIRA>
In-Reply-To: <JIRA.13111993.1508943103000@Atlassian.JIRA>
References: <JIRA.13111993.1508943103000@Atlassian.JIRA> <JIRA.13111993.1508943103692@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-12710) HTTPFS HTTP max header size env
 variable is not respected in branch-2
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 25 Oct 2017 17:54:12 -0000


    [ https://issues.apache.org/jira/browse/HDFS-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219181#comment-16219181 ] 

Istvan Fajth commented on HDFS-12710:
-------------------------------------

Hi [~jzhuge],

Thank you for checking into this JIRA!

Actually I have tested this on a CDH cluster that contained the patch, and we found that after we set the environment variable added in HDFS-10423 it is not applied in the server, we used the same test script to test the CDH cluster that was added to HDFS-10423, and the value in the environment variable was not used by the server. We ran into the default 8192 limit.

When we added some debug prints, we made sure that the environment variable is set, and visible for the script when it runs, and starts HTTPFS.
Then we tracked down that these properties are going to the catalina.properties file, which is being created based on the catalina-default.properties in and this calls to catalina_set_property method in this script to add custom properties to the end of the file.
After we ensured that the catalina.properties file contains the mapping from the environment variable to the property that is used in the server.xml things started to work as expected. We ensured this by adding the same line to the same script, and ran the test script proposed by Nicolae in the original JIRA HDFS-10423.

As the code is the same I did not went to test it with a clean hadoop install without CDH wrappers as it did not felt necessary. Should I, and then add testing results here?

> HTTPFS HTTP max header size env variable is not respected in branch-2
> ---------------------------------------------------------------------
>
>                 Key: HDFS-12710
>                 URL: https://issues.apache.org/jira/browse/HDFS-12710
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.5, 2.7.4, 2.8.1
>            Reporter: Istvan Fajth
>            Assignee: Istvan Fajth
>            Priority: Minor
>         Attachments: HDFS-12710-branch-2.001.patch
>
>
> In HDFS-10423 there is the HTTPFS_MAX_HTTP_HEADER_SIZE environment variable added, and as well the server.xml for the HTTPFS service started to utilize a variable in the config for maxHttpHeaderSize.
> The variable is added in httpfs-config.sh and exported there and as well an example is added to httpfs-env.sh
> The default value is not modified though, the httpfs.sh script has a section in which it sets catalina properties, and there the environment variable is not mapped to the catalina property that is used by the server.xml.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org