hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15546) ABFS: tune imports & javadocs; stabilise tests
Date Fri, 20 Jul 2018 01:00:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550079#comment-16550079

Steve Loughran commented on HADOOP-15546:

HADOOP-15546 patch 008
* review ITestAzureBlobFileSystemListStatus
* move unicode test in ITestAzureBlobFileSystemRename into parameterized test suite ITestAzureBlobFileSystemRenameUnicode;
isolates failures better. (left the old one there for now)
* ITestReadWriteAndSeek -> ITestAbfsReadWriteAndSeek
* helper methods touch(path), write(path, data) in AbstractAbfsIntegrationTest; use where
* all tests of > a megabyte or two tagged as scale, so splitting up "basic fast set" from
"the full diligence set". People must still due that scale test before a final commit, but
it means that basic dev cycles are shorter

Ignoring the parallel test issue (which I am), I'm pretty much done here. If things are failing,
I don't think it's because this patch breaks stuff, it just  surfaces some problems.

Test results
* ITestAzureBlobFileSystemRenameUnicode fails
* All the wasb tests are failing with an NPE inside wasb using the login details which I am
using to run the abfs tests. (as shown above)
* new test which compare LocatedFileStatus to VersionedFileStatus fail if it VersionedFileStatus
is on the left of the .equals(), as it requires a matching type and version. To date, all
FileStatus subclasses just compare on path. *I don't know if this is going to be a problem*.

I think this is ready to go in. Yes, stuff fails, but this gets the tests runnable by others,
so we can make progress on the branch. Reviews please!

> ABFS: tune imports & javadocs; stabilise tests
> ----------------------------------------------
>                 Key: HADOOP-15546
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15546
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: HADOOP-15407
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HADOOP-15546-001.patch, HADOOP-15546-HADOOP-15407-001.patch, HADOOP-15546-HADOOP-15407-002.patch,
HADOOP-15546-HADOOP-15407-003.patch, HADOOP-15546-HADOOP-15407-004.patch, HADOOP-15546-HADOOP-15407-005.patch,
HADOOP-15546-HADOOP-15407-006.patch, HADOOP-15546-HADOOP-15407-006.patch, HADOOP-15546-HADOOP-15407-007.patch,
> Followup on HADOOP-15540 with some initial review tuning
> h2. Tuning
> * ordering of imports
> * rely on azure-auth-keys.xml to store credentials (change imports, docs,.gitignore)
> * log4j -> info
> * add a "." to the first sentence of all the javadocs I noticed.
> * remove @Public annotations except for some constants (which includes some commitment
to maintain them).
> * move the AbstractFS declarations out of the src/test/resources XML file into core-default.xml
for all to use
> * other IDE-suggested tweaks
> h2. Testing
> Review the tests, move to ContractTestUtil assertions, make more consistent to contract
test setup, and general work to make the tests work well over slower links, document, etc.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message