Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 12 Sep 2017 12:53:00 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13100372.1504766523000.90187.1505220780458@Atlassian.JIRA>
In-Reply-To: <JIRA.13100372.1504766523000@Atlassian.JIRA>
References: <JIRA.13100372.1504766523000@Atlassian.JIRA> <JIRA.13100372.1504766523306@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HADOOP-14845) azure getFileStatus not making
 any auth checks
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 12 Sep 2017 12:53:07 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-14845?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16=
162885#comment-16162885 ]=20

Hadoop QA commented on HADOOP-14845:
------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 18s=
{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0=
m  0s{color} | {color:green} The patch does not contain any @author tags. {=
color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}=
  0m  0s{color} | {color:green} The patch appears to include 4 new or modif=
ied test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}=
 13m 43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0=
m 19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}=
  0m 14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0=
m 21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  =
0m 27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0=
m 14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}=
  0m 17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0=
m 16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m =
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:oran=
ge}  0m 13s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch ge=
nerated 3 new + 80 unchanged - 0 fixed =3D 83 total (was 80) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0=
m 19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}=
  0m  0s{color} | {color:green} The patch has no whitespace issues. {color}=
 |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 35s=
{color} | {color:red} hadoop-tools/hadoop-azure generated 1 new + 0 unchang=
ed - 0 fixed =3D 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0=
m 12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  2m  3s{col=
or} | {color:red} hadoop-azure in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}=
  0m 14s{color} | {color:green} The patch does not generate ASF License war=
nings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m  2s{colo=
r} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-tools/hadoop-azure |
|  |  Redundant nullcheck of org.apache.hadoop.fs.azure.NativeAzureFileSyst=
em.getFileStatusInternal(Path), which is known to be non-null in org.apache=
.hadoop.fs.azure.NativeAzureFileSystem.existsInternal(Path)  Redundant null=
 check at NativeAzureFileSystem.java:is known to be non-null in org.apache.=
hadoop.fs.azure.NativeAzureFileSystem.existsInternal(Path)  Redundant null =
check at NativeAzureFileSystem.java:[line 2201] |
| Failed junit tests | hadoop.fs.azure.TestNativeAzureFileSystemConcurrency=
 |
|   | hadoop.fs.azure.TestWasbFsck |
|   | hadoop.fs.azure.TestOutOfBandAzureBlobOperations |
|   | hadoop.fs.azure.TestNativeAzureFileSystemContractMocked |
|   | hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked |
|   | hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck |
|   | hadoop.fs.azure.TestNativeAzureFileSystemMocked |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-14845 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/1288662=
5/HADOOP-14845.001.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsit=
e  unit  findbugs  checkstyle  |
| uname | Linux 82582371a854 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 1=
1:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provide=
d.sh |
| git revision | trunk / e74d1be |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/13262/a=
rtifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt |
| findbugs | https://builds.apache.org/job/PreCommit-HADOOP-Build/13262/art=
ifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html |
| unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/13262/artifac=
t/patchprocess/patch-unit-hadoop-tools_hadoop-azure.txt |
|  Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/1326=
2/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/132=
62/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.


> azure getFileStatus not making any auth checks
> ----------------------------------------------
>
>                 Key: HADOOP-14845
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14845
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>            Reporter: Sivaguru Sankaridurg
>            Assignee: Sivaguru Sankaridurg
>              Labels: azure, fs, secure, wasb
>         Attachments: HADOOP-14845.001.patch
>
>
> The HDFS spec requires only traverse checks for any file accessed via get=
FileStatus ... and since WASB does not support traverse checks, removing th=
is call effectively removed all protections for the getFileStatus call. The=
 reasoning at that time was that doing a performAuthCheck was the wrong thi=
ng to do, since it was going against the spec....and that the correct fix t=
o the getFileStatus issue was to implement traverse checks rather than go a=
gainst the spec by calling performAuthCheck. The side-effects of such a cha=
nge were not fully clear at that time, but the thinking was that it was saf=
er to remain true to the spec, as far as possible.
> The reasoning remains correct even today. But in view of the security hol=
e introduced by this change (that anyone can load up any other user's data =
in hive), and keeping in mind that WASB does not intend to implement traver=
se checks, we propose a compromise.
> We propose (re)introducing a read-access check to getFileStatus(), that w=
ould check the existing ancestor for read-access whenever invoked. Although=
 not perfect (in that it is a departure from the spec), we believe that it =
is a good compromise between having no checks at all; and implementing full=
-blown traverse checks.
> For scenarios that deal with intermediate folders like mkdirs, the call w=
ould check for read access against an existing ancestor (when invoked from =
shell) for intermediate non-existent folders =E2=80=93 {{ mkdirs /foo/bar, =
where only "/" exists, would result in read-checks against "/" for "/","/fo=
o" and "/foo/bar" }}. This can be thought of, as being a close-enough subst=
itute for the traverse checks that hdfs does.
> For other scenarios that don't deal with non-existent intermediate folder=
s =E2=80=93 like read, delete etc, the check will happen against the parent=
. Once again, we can think of the read-check against the parent as a substi=
tute for the traverse check, which can be customized for various users with=
 ranger policies.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org