hadoop-ozone-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sammic...@apache.org
Subject [hadoop-ozone] branch HDDS-1564 updated (d5666c5 -> a92058e)
Date Thu, 13 Feb 2020 08:28:25 GMT
This is an automated email from the ASF dual-hosted git repository.

sammichen pushed a change to branch HDDS-1564
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.


 discard d5666c5  HDDS-2923 Add fall-back protection for rack awareness in pipeline creation.
(#516)
 discard dd2b7e8  HDDS-2924. Fix Pipeline#nodeIdsHash collision issue. (#478)
 discard 8b42cdc   HDDS-2913 Update config names and CLI for multi-raft feature. (#462)
 discard b63a745  HDDS-2772 Better management for pipeline creation limitation. (#410)
 discard cb5486b  HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI
(#375)
 discard facd403  HDDS-2756. Handle pipeline creation failure in different way when it exceeds
pipeline limit
 discard 4d463d6  Resolve rebase conflict.
 discard d6b9ec0  HDDS-1574 Average out pipeline allocation on datanodes and add metrcs/test
(#291)
 discard 6cdfdcb  HDDS-2650 Fix createPipeline CLI and make it message based. (#370)
 discard b5ef117  Revert "HDDS-2650 Fix createPipeline CLI. (#340)"
 discard 81ed727  HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (#348)
 discard e48ec92  HDDS-2650 Fix createPipeline CLI. (#340)
 discard e98c12f  Rebase Fix
 discard 4cbf554  HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN pipeline. (#237)
 discard 1ef5e6d  HDDS-1569 Support creating multiple pipelines with same datanode. Contributed
by Li Cheng.
 discard 3d6e5b1  HDDS-2089: Add createPipeline CLI. (#1418)
 discard 5cbfb37  HDDS-1571. Create an interface for pipeline placement policy to support
network topologies. (#1395)
 discard 6ab3326  HDDS-1577. Add default pipeline placement policy implementation. (#1366)
     add 784acad  HDDS-2876. Consolidate ObjectID and UpdateID from Info objects into one
class
     add 3e7eb1e  HDDS-2972. Any container replication error can terminate SCM service
     add 295d030  HDDS-2975. Print Freon summary to log in non-interactive mode
     add fb90756  HDDS-2946. Rename audit log should contain both srcKey and dstKey not just
key
     add 92a668a  HDDS-2960. Collect output of crashed tests
     add c654a83  HDDS-2958. Handle replay of OM Volume ACL requests (#510)
     add d3f8d51  HDDS-2959. Handle replay of OM Key ACL requests (#512)
     add 54c4adf  HDDS-2936. Hive queries fail at readFully. (#507)
     add 322120c  HDDS-2944. Handle replay of KeyCommitRequest and DirectoryCreateRequest
(#501)
     add 3ddcdbb  HDDS-2953. Handle replay of S3 requests (#504)
     add 51735f4  HDDS-2987. Add metrics to OM DoubleBuffer
     add 9a49388  HDDS-2891. Apache NiFi PutFile processor is failing with secure Ozone S3G
(#449)
     add b2e8966  HDDS-2962. Handle replay of OM Prefix ACL requests (#513)
     add 25d2338  HDDS-2914. Certain Hive queries started to fail on generating splits (#540)
     add 94f81be  HDDS-2790. concept/Overview.md translation (#497)
     add d48e7f7  HDDS-1577. Add default pipeline placement policy implementation. (#1366)
     add 260938e  HDDS-1571. Create an interface for pipeline placement policy to support
network topologies. (#1395)
     add 7576f4d  HDDS-2089: Add createPipeline CLI. (#1418)
     add 3749f40  HDDS-1569 Support creating multiple pipelines with same datanode. Contributed
by Li Cheng.
     add e1b168f  HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN pipeline. (#237)
     add 41a30ca  Rebase Fix
     add 90f794f  HDDS-2650 Fix createPipeline CLI. (#340)
     add e720e7a  HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (#348)
     add 5c6d412  Revert "HDDS-2650 Fix createPipeline CLI. (#340)"
     add 1a27b25  HDDS-2650 Fix createPipeline CLI and make it message based. (#370)
     add 0bd5714  HDDS-1574 Average out pipeline allocation on datanodes and add metrcs/test
(#291)
     add 605f960  Resolve rebase conflict.
     add 8f57dbb  HDDS-2756. Handle pipeline creation failure in different way when it exceeds
pipeline limit
     add 6613b05  HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI
(#375)
     add 71aa879  HDDS-2772 Better management for pipeline creation limitation. (#410)
     add 484ab00   HDDS-2913 Update config names and CLI for multi-raft feature. (#462)
     add 8211fcd  HDDS-2924. Fix Pipeline#nodeIdsHash collision issue. (#478)
     add a92058e  HDDS-2923 Add fall-back protection for rack awareness in pipeline creation.
(#516)

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (d5666c5)
            \
             N -- N -- N   refs/heads/HDDS-1564 (a92058e)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/hadoop/ozone/OzoneConsts.java  |   2 +
 .../docs/content/concept/{Hdds.md => Hdds.zh.md}   |   0
 hadoop-hdds/docs/content/concept/Overview.zh.md    |  50 +++++
 .../{OzoneManager.md => OzoneManager.zh.md}        |   0
 .../hdds/scm/container/ReplicationManager.java     |   2 +
 .../hdds/scm/pipeline/RatisPipelineUtils.java      |   1 +
 .../hadoop/ozone/client/io/KeyOutputStream.java    |  47 +++--
 .../apache/hadoop/ozone/client/rpc/RpcClient.java  |   2 +
 .../main/java/org/apache/hadoop/ozone/OmUtils.java |  12 +-
 .../hadoop/ozone/om/exceptions/OMException.java    |   4 +
 .../hadoop/ozone/om/helpers/OmBucketInfo.java      |  64 ++-----
 .../apache/hadoop/ozone/om/helpers/OmKeyInfo.java  |  61 ++-----
 .../ozone/om/helpers/OmMultipartKeyInfo.java       |  35 +---
 .../hadoop/ozone/om/helpers/OmPrefixInfo.java      |  22 ++-
 .../hadoop/ozone/om/helpers/OmVolumeArgs.java      |  59 ++----
 .../hadoop/ozone/om/helpers/OzoneFileStatus.java   |  14 +-
 .../hadoop/ozone/om/helpers/WithObjectID.java      |  92 ++++++++++
 ...OzoneManagerProtocolClientSideTranslatorPB.java |  11 +-
 .../src/main/proto/OzoneManagerProtocol.proto      |   6 +
 .../hadoop/ozone/om/helpers/TestOmPrefixInfo.java  |   2 +-
 .../dev-support/checks/_mvn_unit_report.sh         |  26 ++-
 hadoop-ozone/dev-support/checks/integration.sh     |   4 +-
 hadoop-ozone/dev-support/checks/unit.sh            |   9 +-
 .../dist/src/main/smoketest/s3/webui.robot         |   3 +-
 .../hadoop/fs/ozone/TestOzoneFileInterfaces.java   |  21 +++
 .../rpc/TestCloseContainerHandlingByClient.java    |   1 -
 .../client/rpc/TestOzoneRpcClientAbstract.java     |   2 +-
 .../org/apache/hadoop/ozone/om/KeyManagerImpl.java |  67 ++++---
 .../apache/hadoop/ozone/om/PrefixManagerImpl.java  |  42 +++--
 .../ozone/om/ratis/OzoneManagerDoubleBuffer.java   |   8 +
 .../metrics/OzoneManagerDoubleBufferMetrics.java   |  29 +++
 .../hadoop/ozone/om/request/OMClientRequest.java   |  22 ++-
 .../om/request/bucket/OMBucketCreateRequest.java   |   3 +-
 .../om/request/bucket/OMBucketDeleteRequest.java   |   3 +-
 .../request/bucket/OMBucketSetPropertyRequest.java |   3 +-
 .../om/request/bucket/acl/OMBucketAclRequest.java  |   3 +-
 .../om/request/file/OMDirectoryCreateRequest.java  | 128 +++++++++----
 .../ozone/om/request/file/OMFileCreateRequest.java |   2 +-
 .../om/request/key/OMAllocateBlockRequest.java     |   4 +-
 .../ozone/om/request/key/OMKeyCommitRequest.java   |  86 ++++++---
 .../ozone/om/request/key/OMKeyCreateRequest.java   |   7 +-
 .../ozone/om/request/key/OMKeyDeleteRequest.java   |   3 +-
 .../ozone/om/request/key/OMKeyPurgeRequest.java    |   2 +-
 .../ozone/om/request/key/OMKeyRenameRequest.java   |  16 +-
 .../hadoop/ozone/om/request/key/OMKeyRequest.java  |  12 --
 .../ozone/om/request/key/acl/OMKeyAclRequest.java  |  61 +++++--
 .../om/request/key/acl/OMKeyAddAclRequest.java     |  51 +++---
 .../om/request/key/acl/OMKeyRemoveAclRequest.java  |  47 ++---
 .../om/request/key/acl/OMKeySetAclRequest.java     |  40 ++--
 .../request/key/acl/prefix/OMPrefixAclRequest.java |  91 ++++++----
 .../key/acl/prefix/OMPrefixAddAclRequest.java      |  51 ++++--
 .../key/acl/prefix/OMPrefixRemoveAclRequest.java   |  48 +++--
 .../key/acl/prefix/OMPrefixSetAclRequest.java      |  44 +++--
 .../request/s3/bucket/S3BucketCreateRequest.java   |   3 +-
 .../request/s3/bucket/S3BucketDeleteRequest.java   |   3 +-
 .../S3InitiateMultipartUploadRequest.java          |  61 ++++---
 .../multipart/S3MultipartUploadAbortRequest.java   |  99 ++++++----
 .../S3MultipartUploadCommitPartRequest.java        |  95 +++++++---
 .../S3MultipartUploadCompleteRequest.java          |  63 +++++--
 .../om/request/volume/OMVolumeCreateRequest.java   |   3 +-
 .../om/request/volume/OMVolumeDeleteRequest.java   |   3 +-
 .../om/request/volume/OMVolumeSetOwnerRequest.java |   3 +-
 .../om/request/volume/OMVolumeSetQuotaRequest.java |   3 +-
 .../om/request/volume/acl/OMVolumeAclRequest.java  |  68 ++++---
 .../request/volume/acl/OMVolumeAddAclRequest.java  |  37 ++--
 .../volume/acl/OMVolumeRemoveAclRequest.java       |  37 ++--
 .../request/volume/acl/OMVolumeSetAclRequest.java  |  35 ++--
 .../response/file/OMDirectoryCreateResponse.java   |  24 +--
 .../om/response/file/OMFileCreateResponse.java     |   5 +
 .../ozone/om/response/key/OMKeyCommitResponse.java |  45 ++---
 .../ozone/om/response/key/OMKeyDeleteResponse.java |   2 +-
 .../om/response/key/acl/OMKeyAclResponse.java      |  29 +--
 .../key/acl/prefix/OMPrefixAclResponse.java        |  46 +++--
 .../S3InitiateMultipartUploadResponse.java         |  39 ++--
 .../multipart/S3MultipartUploadAbortResponse.java  |  60 +++---
 .../S3MultipartUploadCommitPartResponse.java       |  32 ++--
 .../S3MultipartUploadCompleteResponse.java         |  64 ++++---
 .../om/response/volume/OMVolumeAclOpResponse.java  |  28 +--
 .../protocolPB/OzoneManagerRequestHandler.java     |   2 +
 ...tOzoneManagerDoubleBufferWithDummyResponse.java |   2 +
 .../ozone/om/request/TestOMRequestUtils.java       |   3 +-
 .../bucket/TestOMBucketSetPropertyRequest.java     |   4 +-
 .../request/file/TestOMDirectoryCreateRequest.java |   2 +-
 .../ozone/om/request/key/TestOMKeyAclRequest.java  |  92 ++++++++++
 .../om/request/key/TestOMKeyCommitRequest.java     |  42 ++++-
 .../om/request/key/TestOMPrefixAclRequest.java     | 101 +++++++++++
 .../volume/acl/TestOMVolumeAddAclRequest.java      |  33 ++++
 .../volume/acl/TestOMVolumeRemoveAclRequest.java   |  47 ++++-
 .../volume/acl/TestOMVolumeSetAclRequest.java      |  35 ++++
 .../file/TestOMDirectoryCreateResponse.java        |  37 +---
 .../om/response/key/TestOMKeyCommitResponse.java   |  11 +-
 .../s3/multipart/TestS3MultipartResponse.java      |   8 +-
 .../fs/ozone/BasicOzoneClientAdapterImpl.java      |  66 ++++++-
 .../hadoop/fs/ozone/BasicOzoneFileSystem.java      |  22 ++-
 .../apache/hadoop/fs/ozone/FileStatusAdapter.java  |  12 +-
 .../hadoop/fs/ozone/FilteredClassLoader.java       |   1 +
 ...uthParser.java => AWSV4SignatureProcessor.java} | 201 +++++++++++++++++----
 .../apache/hadoop/ozone/s3/HeaderPreprocessor.java |  16 +-
 .../hadoop/ozone/s3/OzoneClientProducer.java       |  84 ++++-----
 .../hadoop/ozone/s3/RootPageDisplayFilter.java     |  40 ++--
 ...{AWSAuthParser.java => SignatureProcessor.java} |  28 +--
 .../hadoop/ozone/s3/VirtualHostStyleFilter.java    |  10 -
 .../hadoop/ozone/s3/endpoint/BucketEndpoint.java   |   5 +-
 .../hadoop/ozone/s3/endpoint/EndpointBase.java     |  13 +-
 .../hadoop/ozone/s3/endpoint/RootEndpoint.java     |  17 +-
 .../s3/header/AuthenticationHeaderParser.java      |  69 -------
 .../ozone/s3/header/AuthorizationHeaderV4.java     |  30 +--
 .../apache/hadoop/ozone/s3/util/OzoneS3Util.java   |   4 +-
 .../resources/webapps/s3gateway/WEB-INF/web.xml    |  10 +-
 .../ozone/s3/TestAWSV4SignatureProcessor.java      | 103 +++++++++++
 .../hadoop/ozone/s3/TestOzoneClientProducer.java   |  13 +-
 .../ozone/s3/TestVirtualHostStyleFilter.java       |  17 +-
 .../hadoop/ozone/s3/endpoint/TestBucketPut.java    |  30 +--
 .../hadoop/ozone/s3/endpoint/TestRootList.java     |  27 ++-
 .../ozone/s3/header/TestAuthorizationHeaderV4.java |   2 +-
 .../hadoop/ozone/freon/BaseFreonGenerator.java     |  27 ++-
 .../java/org/apache/hadoop/ozone/freon/Freon.java  |   5 +
 .../org/apache/hadoop/ozone/freon/ProgressBar.java |  10 +-
 118 files changed, 2361 insertions(+), 1232 deletions(-)
 copy hadoop-hdds/docs/content/concept/{Hdds.md => Hdds.zh.md} (100%)
 create mode 100644 hadoop-hdds/docs/content/concept/Overview.zh.md
 copy hadoop-hdds/docs/content/concept/{OzoneManager.md => OzoneManager.zh.md} (100%)
 create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/WithObjectID.java
 create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyAclRequest.java
 create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMPrefixAclRequest.java
 rename hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/{AWSV4AuthParser.java
=> AWSV4SignatureProcessor.java} (67%)
 copy hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/NoCacheFilter.java
=> hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/RootPageDisplayFilter.java
(57%)
 rename hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/{AWSAuthParser.java
=> SignatureProcessor.java} (80%)
 delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthenticationHeaderParser.java
 create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestAWSV4SignatureProcessor.java


---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org


Mime
View raw message