hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15546) ABFS: tune imports & javadocs; stabilise tests
Date Fri, 20 Jul 2018 01:00:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550079#comment-16550079
] 

Steve Loughran commented on HADOOP-15546:
-----------------------------------------

HADOOP-15546 patch 008
* review ITestAzureBlobFileSystemListStatus
* move unicode test in ITestAzureBlobFileSystemRename into parameterized test suite ITestAzureBlobFileSystemRenameUnicode;
isolates failures better. (left the old one there for now)
* ITestReadWriteAndSeek -> ITestAbfsReadWriteAndSeek
* helper methods touch(path), write(path, data) in AbstractAbfsIntegrationTest; use where
appropriate
* all tests of > a megabyte or two tagged as scale, so splitting up "basic fast set" from
"the full diligence set". People must still due that scale test before a final commit, but
it means that basic dev cycles are shorter

Ignoring the parallel test issue (which I am), I'm pretty much done here. If things are failing,
I don't think it's because this patch breaks stuff, it just  surfaces some problems.

Test results
* ITestAzureBlobFileSystemRenameUnicode fails
* All the wasb tests are failing with an NPE inside wasb using the login details which I am
using to run the abfs tests. (as shown above)
* new test which compare LocatedFileStatus to VersionedFileStatus fail if it VersionedFileStatus
is on the left of the .equals(), as it requires a matching type and version. To date, all
FileStatus subclasses just compare on path. *I don't know if this is going to be a problem*.


I think this is ready to go in. Yes, stuff fails, but this gets the tests runnable by others,
so we can make progress on the branch. Reviews please!



> ABFS: tune imports & javadocs; stabilise tests
> ----------------------------------------------
>
>                 Key: HADOOP-15546
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15546
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: HADOOP-15407
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HADOOP-15546-001.patch, HADOOP-15546-HADOOP-15407-001.patch, HADOOP-15546-HADOOP-15407-002.patch,
HADOOP-15546-HADOOP-15407-003.patch, HADOOP-15546-HADOOP-15407-004.patch, HADOOP-15546-HADOOP-15407-005.patch,
HADOOP-15546-HADOOP-15407-006.patch, HADOOP-15546-HADOOP-15407-006.patch, HADOOP-15546-HADOOP-15407-007.patch,
HADOOP-15546-HADOOP-15407-008.patch
>
>
> Followup on HADOOP-15540 with some initial review tuning
> h2. Tuning
> * ordering of imports
> * rely on azure-auth-keys.xml to store credentials (change imports, docs,.gitignore)
> * log4j -> info
> * add a "." to the first sentence of all the javadocs I noticed.
> * remove @Public annotations except for some constants (which includes some commitment
to maintain them).
> * move the AbstractFS declarations out of the src/test/resources XML file into core-default.xml
for all to use
> * other IDE-suggested tweaks
> h2. Testing
> Review the tests, move to ContractTestUtil assertions, make more consistent to contract
test setup, and general work to make the tests work well over slower links, document, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message