hadoop-ozone-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From msi...@apache.org
Subject [hadoop-ozone] branch HDDS-2886 updated (471d203 -> 1cc9d1e)
Date Mon, 09 Mar 2020 14:15:18 GMT
This is an automated email from the ASF dual-hosted git repository.

msingh pushed a change to branch HDDS-2886
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.


 discard 471d203  fix checkstyle issues
 discard 064c59d  fix review comments
 discard 2b8e449  HDDS-2886. parse and dump datanode segment file to pritable text
     add f160f46  HDDS-3059 Changed auditing in OzoneManager#getFileStatus to read (#587)
     add c9c4b9b  HDDS-3026 Changed auditing in OzoneManager#listStatus to read (#585)
     add e41f10e  HDDS-1590 Aggregate bytes written metrics from container to datanode and
to SCM (#393)
     add 6b50bf9  HDDS-3052. Test ChunkManagerImpl performance with long-running freon tests.
(#580)
     add 4bdd600  HDDS-3044. Fix TestDeleteWithSlowFollower.java. (#574)
     add 03094c4  Ignore failing tests: TestRatisPipelineLeader, TestRatisPipelineProvider
     add bb79045  HDDS-3024. README is missing from the source release tar
     add c4812d5  HDDS-3043. Fix TestFailureHandlingByClient.java. (#591)
     add 8ef5c39  HDDS-3046. Fix Retry handling in Rpc Client. (#575)
     add 302ca84  HDDS-3063. Add test to verify replication factor of ozone fs (#593)
     add 63e6ac2  HDDS-3065. Ozone Filesystem should return real default replication (#595)
     add 39b8ea8  HDDS-3055. SCM crash during startup does not print any error message to
log. (#581)
     add cfd12f2  HDDS-3068. OM crash during startup does not print any error message to log.
(#599)
     add 3b73907  HDDS-3034. Broken return code check in unit/integration
     add 40ce72f  HDDS-2984. Allocate Block failing with NPE
     add 18c48ca  HDDS-3025. Fail to connect to s3g by http://ip:port
     add 980b7cf  HDDS-2648. TestOzoneManagerDoubleBufferWithOMResponse. (#600)
     add c5a5172  HDDS-3067. Fix Bug in Scrub Pipeline causing destory pipelines after SCM
restart. (#598)
     add 1d47bb1  HDDS-3070. NPE when stop recon server while recon server was not really
started before. (#603)
     add 551d149  HDDS-2996. Create REST API to serve Node information and integrate with
UI in Recon.
     add 8f3a42f  HDDS-2974. Create Freon test to test isolated Ratis FOLLOWER
     add 28f2629  HDDS-2940. mkdir : create key table entries for intermediate directories
in the path (#498)
     add 25baca3  HDDS-3038. TestRatisPipelineLeader fails since we no longer wait for leader
in the HealthyPipelineSafeModeExitRule. (#604)
     add 912dcf0  HDDS-2816. Fix shell description for --start parameter of listing keys (#601)
     add 65bc07e  HDDS-2799. beyond/DockerCheatSheet.md translation
     add b441954  HDDS-3066. SCM crash during loading containers to DB. (#596)
     add cc16844  HDDS-3054. OzoneFileStatus#getModificationTime should return actual directory
modification time when its OmKeyInfo is available (#584)
     add 5bb2a96  HDDS-3085. OM Delta updates request in Recon should work with secure Ozone
Manager. (#615)
     add f9637ac  HDDS-3109. Refactor 'Recon' in MiniOzoneCluster to use ephemeral port. (#617)
     add f5ea42a  HDDS-2716. Add integration test to verify pipeline closed on read statemachine
failure (#576)
     add 9a72c6d  HDDS-2886. parse and dump datanode segment file to pritable text
     add 826909c  fix review comments
     add 1cc9d1e  fix checkstyle issues

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (471d203)
            \
             N -- N -- N   refs/heads/HDDS-2886 (1cc9d1e)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 .../apache/hadoop/hdds/recon/ReconConfigKeys.java  |   4 +
 .../common/src/main/resources/ozone-default.xml    |   4 +-
 .../apache/hadoop/ozone/HddsDatanodeService.java   |  40 ++
 .../transport/server/ratis/XceiverServerRatis.java |  21 +-
 .../background/BlockDeletingService.java           |   2 +-
 .../proto/StorageContainerDatanodeProtocol.proto   |   1 +
 .../docs/content/beyond/DockerCheatSheet.zh.md     |  85 ++++
 .../hdds/scm/container/SCMContainerManager.java    |  39 +-
 .../hadoop/hdds/scm/node/NodeStateManager.java     |   1 +
 .../hdds/scm/pipeline/PipelineReportHandler.java   |  14 +-
 .../hdds/scm/pipeline/SCMPipelineManager.java      |  47 ++-
 .../hdds/scm/pipeline/SCMPipelineMetrics.java      |  24 +-
 .../scm/safemode/HealthyPipelineSafeModeRule.java  |  18 +-
 .../scm/server/StorageContainerManagerStarter.java |   9 +-
 .../scm/pipeline/MockRatisPipelineProvider.java    |  17 +-
 .../scm/pipeline/TestRatisPipelineProvider.java    |  29 +-
 .../safemode/TestHealthyPipelineSafeModeRule.java  |  19 +-
 .../hdds/scm/safemode/TestSCMSafeModeManager.java  |   6 +-
 .../org/apache/hadoop/ozone/om/OMConfigKeys.java   |   6 +
 .../hadoop/ozone/om/helpers/OzoneFileStatus.java   |   2 +-
 ...OzoneManagerProtocolClientSideTranslatorPB.java |  27 +-
 hadoop-ozone/dev-support/checks/integration.sh     |   3 +
 hadoop-ozone/dev-support/checks/unit.sh            |   3 +
 hadoop-ozone/dev-support/intellij/ozone-site.xml   |   8 +
 .../dist/src/main/assemblies/ozone-src.xml         |   4 +-
 .../dist/src/main/compose/ozone/docker-config      |   2 +-
 hadoop-ozone/dist/src/main/compose/ozone/test.sh   |   2 +
 .../src/main/compose/ozonesecure/docker-config     |   8 +-
 .../dist/src/main/compose/ozonesecure/test.sh      |   3 +-
 .../dist/src/main/smoketest/ozonefs/ozonefs.robot  |   2 +
 .../dist/src/main/smoketest/recon/recon-api.robot  |  15 +-
 .../hadoop/fs/ozone/TestOzoneFileInterfaces.java   | 108 ++++-
 .../hadoop/fs/ozone/TestOzoneFileSystem.java       |  46 ++
 .../hadoop/hdds/scm/TestRatisPipelineLeader.java   |  65 ++-
 .../org/apache/hadoop/ozone/MiniOzoneCluster.java  |  30 ++
 .../apache/hadoop/ozone/MiniOzoneClusterImpl.java  |  91 ++--
 .../hadoop/ozone/TestOzoneConfigurationFields.java |   1 +
 .../TestContainerStateMachineFailureOnRead.java    | 224 ++++++++++
 .../client/rpc/TestDeleteWithSlowFollower.java     |  35 +-
 .../client/rpc/TestFailureHandlingByClient.java    |  25 +-
 ...stRecon.java => TestReconWithOzoneManager.java} |  29 +-
 .../TestSCMPipelineBytesWrittenMetrics.java        | 127 ++++++
 .../org/apache/hadoop/ozone/om/KeyManagerImpl.java |  88 ++--
 .../org/apache/hadoop/ozone/om/OzoneManager.java   |  32 +-
 .../hadoop/ozone/om/OzoneManagerStarter.java       |   9 +-
 .../om/request/bucket/OMBucketCreateRequest.java   |   4 +-
 .../om/request/file/OMDirectoryCreateRequest.java  | 131 +++++-
 .../ozone/om/request/file/OMFileCreateRequest.java |   4 +-
 .../ozone/om/request/file/OMFileRequest.java       | 108 ++++-
 .../hadoop/ozone/om/request/key/OMKeyRequest.java  |   5 +-
 .../request/s3/bucket/S3BucketCreateRequest.java   |   4 +-
 .../S3InitiateMultipartUploadRequest.java          |   6 +-
 .../om/request/volume/OMVolumeCreateRequest.java   |   4 +-
 .../response/file/OMDirectoryCreateResponse.java   |  28 +-
 .../ozone/web/ozShell/keys/ListKeyHandler.java     |   3 +-
 ...TestOzoneManagerDoubleBufferWithOMResponse.java |  11 +-
 .../request/volume/TestOMVolumeCreateRequest.java  |  20 +-
 .../file/TestOMDirectoryCreateResponse.java        |   2 +-
 .../fs/ozone/BasicOzoneClientAdapterImpl.java      |   5 +
 .../hadoop/fs/ozone/BasicOzoneFileSystem.java      |   5 +
 .../apache/hadoop/fs/ozone/OzoneClientAdapter.java |   2 +
 .../fs/ozone/TestOzoneFileSystemWithMocks.java     | 120 +++---
 hadoop-ozone/recon/pom.xml                         |  24 ++
 .../apache/hadoop/ozone/recon/ReconHttpServer.java |   2 +-
 .../org/apache/hadoop/ozone/recon/ReconServer.java |  58 ++-
 .../hadoop/ozone/recon/ReconServerConfigKeys.java  |   4 +-
 .../hadoop/ozone/recon/api/NodeEndpoint.java       | 113 +++++
 .../ozone/recon/api/types/ContainersResponse.java  |  11 -
 .../ozone/recon/api/types/DatanodeMetadata.java    |  89 ++++
 .../recon/api/types/DatanodeStorageReport.java}    |  35 +-
 .../ozone/recon/api/types/DatanodesResponse.java}  |  41 +-
 .../hadoop/ozone/recon/api/types/KeysResponse.java |  11 -
 .../ozone/recon/scm/ReconContainerManager.java     |   9 +-
 .../hadoop/ozone/recon/scm/ReconNodeManager.java   |  37 +-
 .../scm/ReconStorageContainerManagerFacade.java    |   9 +-
 .../ozone/recon/tasks/ReconTaskControllerImpl.java |   4 +-
 .../webapps/recon/ozone-recon-web/LICENSE          |  32 ++
 .../webapps/recon/ozone-recon-web/api/db.json      | 162 +++----
 .../webapps/recon/ozone-recon-web/package.json     |   3 +-
 .../webapps/recon/ozone-recon-web/src/App.less     |   6 +
 .../webapps/recon/ozone-recon-web/src/App.tsx      |   4 +-
 .../recon/ozone-recon-web/src/utils/themeIcons.tsx |  27 +-
 .../src/views/Datanodes/Datanodes.less             |  18 +-
 .../src/views/Datanodes/Datanodes.tsx              |  67 +--
 .../webapps/recon/ozone-recon-web/yarn.lock        |  13 +-
 .../ozone/recon/AbstractOMMetadataManagerTest.java |  11 +-
 .../hadoop/ozone/recon/api/TestNodeEndpoint.java   | 261 ++++++++++++
 .../recon/types/GuiceInjectorUtilsForTests.java    |   3 +
 .../hadoop/ozone/s3/AWSV4SignatureProcessor.java   |   3 +-
 .../ozone/s3/TestAWSV4SignatureProcessor.java      |   4 +-
 .../hadoop/ozone/freon/BaseAppendLogGenerator.java |  60 +++
 .../hadoop/ozone/freon/BaseFreonGenerator.java     |   7 +
 .../hadoop/ozone/freon/ChunkManagerDiskWrite.java  | 165 ++++++++
 .../freon/FollowerAppendLogEntryGenerator.java     | 468 +++++++++++++++++++++
 .../java/org/apache/hadoop/ozone/freon/Freon.java  |   6 +-
 95 files changed, 3047 insertions(+), 557 deletions(-)
 create mode 100644 hadoop-hdds/docs/content/beyond/DockerCheatSheet.zh.md
 create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineFailureOnRead.java
 rename hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/{TestRecon.java
=> TestReconWithOzoneManager.java} (94%)
 create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/pipeline/TestSCMPipelineBytesWrittenMetrics.java
 create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/NodeEndpoint.java
 create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/DatanodeMetadata.java
 copy hadoop-ozone/{common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadList.java
=> recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/DatanodeStorageReport.java}
(61%)
 copy hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetricsInfo.java
=> recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/DatanodesResponse.java}
(52%)
 copy hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/InsufficientDatanodesException.java
=> hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/utils/themeIcons.tsx
(68%)
 create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestNodeEndpoint.java
 create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/BaseAppendLogGenerator.java
 create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/ChunkManagerDiskWrite.java
 create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/FollowerAppendLogEntryGenerator.java


---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-commits-help@hadoop.apache.org


Mime
View raw message