hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Istvan Fajth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12710) HTTPFS HTTP max header size env variable is not respected in branch-2
Date Wed, 25 Oct 2017 17:54:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219181#comment-16219181

Istvan Fajth commented on HDFS-12710:

Hi [~jzhuge],

Thank you for checking into this JIRA!

Actually I have tested this on a CDH cluster that contained the patch, and we found that after
we set the environment variable added in HDFS-10423 it is not applied in the server, we used
the same test script to test the CDH cluster that was added to HDFS-10423, and the value in
the environment variable was not used by the server. We ran into the default 8192 limit.

When we added some debug prints, we made sure that the environment variable is set, and visible
for the script when it runs, and starts HTTPFS.
Then we tracked down that these properties are going to the catalina.properties file, which
is being created based on the catalina-default.properties in and this calls to catalina_set_property
method in this script to add custom properties to the end of the file.
After we ensured that the catalina.properties file contains the mapping from the environment
variable to the property that is used in the server.xml things started to work as expected.
We ensured this by adding the same line to the same script, and ran the test script proposed
by Nicolae in the original JIRA HDFS-10423.

As the code is the same I did not went to test it with a clean hadoop install without CDH
wrappers as it did not felt necessary. Should I, and then add testing results here?

> HTTPFS HTTP max header size env variable is not respected in branch-2
> ---------------------------------------------------------------------
>                 Key: HDFS-12710
>                 URL: https://issues.apache.org/jira/browse/HDFS-12710
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.5, 2.7.4, 2.8.1
>            Reporter: Istvan Fajth
>            Assignee: Istvan Fajth
>            Priority: Minor
>         Attachments: HDFS-12710-branch-2.001.patch
> In HDFS-10423 there is the HTTPFS_MAX_HTTP_HEADER_SIZE environment variable added, and
as well the server.xml for the HTTPFS service started to utilize a variable in the config
for maxHttpHeaderSize.
> The variable is added in httpfs-config.sh and exported there and as well an example is
added to httpfs-env.sh
> The default value is not modified though, the httpfs.sh script has a section in which
it sets catalina properties, and there the environment variable is not mapped to the catalina
property that is used by the server.xml.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message