hadoop-ozone-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei-Chiu Chuang <weic...@cloudera.com.INVALID>
Subject Notes from Hadoop storage community online sync
Date Thu, 07 Nov 2019 18:37:30 GMT
Thanks @Xiaoyu Yao <xyao@cloudera.com> for giving us a great status update
on Ozone!

We had a pretty large group yesterday. Here's my notes for your reference:
~20 contributors joined the discussion.Weichiu, Xiaoyu, Chen, Haihua,
haiyang, hexiaoqiao, Hui, Jinglun, Li, Lisheng, Oliver, sibyl.lv, Sammi,
Yisheng, aiphago, Dazhuang, haicai and many others.
Xiaoyu led the discussion of Ozone: object store for big data workloads.What
and why, feature set, current development: 0.4 features (security) and 0.5
features (HA), future roadmap: scale and stability improvement.

Decommissioning support in progress



   Python client implementation — S3 or RPC

      Sammi: Tencent is preparing to introduce Ozone at Tencent. Use case
      1: Hive. Use case 2: Data science use cases, small files. Requires Python

   Ozone GA timeline

   How does client read: is OM involved in reading data? Ans: No. client
   access DataNode directly.

   What metadata does OM and SCM maintain?

   When can Ozone be used in production environment? Ans: wait for GA, and
   benchmarks running workloads like TPC-DS.

   Performance comparison between HDFS and Ozone. Ans: Ozone use RocksDB as
   the persistent store for metadata, and optimization and tuning is required
   for RocksDB.

   Ozone uses Raft replication protocol. What if it replicates more than 3
   copies? Would the leader become the bottleneck? Ans: multi Raft project is
   undergoing which addresses this problem.

   Rename? Ozone is flat hierarchy. Does it mean rename is a O(n)
   operation? Ans: Ozone plans to support hierarchy.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message