pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] sijie opened a new pull request #2398: [tiered storage] store driver name and driver specific metadata in original ledger metadata
Date Fri, 17 Aug 2018 20:59:57 GMT
sijie opened a new pull request #2398:  [tiered storage] store driver name and driver specific
metadata in original ledger metadata
URL: https://github.com/apache/incubator-pulsar/pull/2398
 
 
    ### Motivation
   
   1) Currently the location of an offloaded ledger isn't stored in the original ledger metadata.
   That means if configuration is changed or modified by mistake. We might potentially cause
data loss.
   
   2) The location of an offloaded ledger is needed by Pulsar SQL. so it is very inconvinient
to
   have the location information stored in a configuration and the approach is also problematic.
   
    ### Changes
   
   Store `driverName` and driver-specific metadata (e.g. bucket name, region name, endpoint)
in the
   original ledger metadata. Change ManagedLedgerImpl to use the driver-specific metadata
to read
   the offloaded ledger. If the driver-specific metadata is missed, it will fall back to use
the configuration.
   
    ### Tests
   
   This change doesn't change the behavior. Existing unit tests and integration tests already
covered the logic.
   
    ### NOTES
   
   Currently the driver name in metadata is not used. We need to use driver name to load different
offloader driver
   after #2393 is implemented

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message