hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng, Kai" <kai.zh...@intel.com>
Subject A top container module like hadoop-cloud for cloud integration modules
Date Mon, 13 Jun 2016 13:02:03 GMT

Noticed it's an obvious trend Hadoop is supporting more and more cloud platforms, I suggest
we have a top container module to hold such integration modules, like the ones for aws, openstack,
azure and upcoming one aliyun. The rational is simple besides the trend:

1.       Existing modules are mixed in Hadoop-tools that becomes a little big being of 18
modules now. Cloud specific ones can be grouped together and separated out, making more sense;

2.       Future abstraction and common specs & codes sharing could be easier or thereafter

3.       Common testing approach could be defined together, for example, some mechanisms as
discussed by Chris, Steve and Allen in HADOOP-12756;

4.       Documentation for "Hadoop on Cloud"? Not sure it's needed, as we already have a section
for "Hadoop compatible File Systems".

If sounds good, the change would be a good fit for Hadoop 3.0, even though the change should
not involve big impact, as it can avoid affecting the artifacts. It may cause some inconveniences
for the current development efforts, though.

Comments are welcome. Thanks!


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message