hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-1521) Use more natural directory structure for ozone
Date Tue, 14 May 2019 09:28:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839254#comment-16839254

Elek, Marton commented on HDDS-1521:

We have no native libraries or man pages but we have smoketest and example deployment for
compose and kubernetes. I am not sure if they can be applied easily to the FHS. And I am not
sure if it's a very frequent use case to extract the tar file directly to the /usr.

It's also a possible approach to keep the tar structure human friendly (and not FHS friendly)
and provide rpm/deb packages (and containers) which are compatible with FHS.

If I understood well the current technical arguments to follow the old structure:

 * It's possible to extract the files directly to /usr 
 * binary path can be referenced without additional setup in PATH variables for the seamless
integration scenario. (In case of using strip for tar)


 * From usability point of view it can be better to provide better naming (which won't be
compatible with FHS)

I am not sure about the following arguments.

bq. log files can be redirect to a different disk than executable binaries to improve some
IO performance.

I think it can be done either way.

bq. ozone dependent jar files can be referenced from a standalone location.

I don't know what is the 'standalone' location but jar files can be referenced from anywhere
without any technical difficulties.

bq. man page can be found at expected location.

I don't think we have man pages and I they are not planned IMHO.

> Use more natural directory structure for ozone
> ----------------------------------------------
>                 Key: HDDS-1521
>                 URL: https://issues.apache.org/jira/browse/HDDS-1521
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>            Reporter: Elek, Marton
>            Priority: Major
> The conversation of the current ozone directory structure is started here: HDDS-1458
> By [~eyang]
> {quote}The proposal is to generate test artifacts in ozone-[version]/share/ozone/tests
for fault injection test framework in release binary tarball.
> {quote}
> By [~anu]:
> {quote}Let us drop the share and make it tests/blockade, and tests/smoketests etc. That
way, all tests that we ship can be found easily. Otherwise I am +1 on this change.
> {quote}
> By [~elek]
> {quote}I agree with Anu. I think It's time to revisit the current structure of ozone
distribution tar files. The original structure is inherited from HADOOP-6255 which is defined
for a project which includes multiple subproject (hadoop, yarn,...) and should support the
creation of different RPM package from the same tar. I think for ozone we can improve the
usability with reorganizing the dirs to a better structure.
> But I am fine to do it in a separated jira and keep it in the share/test until that to
make progress.
> {quote}
> By [~eyang]
> {quote}The reason for HADOOP-6255 was more than just RPM packaging. The motivation behind
the reorganization was to make a directory structure that can work as standalone tarball as
well as follow the general guideline for [Filesystem Hierarchy Standard|https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard].
This was proposed because a several people saw the need to make the structure more unix like
for sharing dependencies for larger eco-system to work. This is the reason that Hadoop HDFS,
Mapreduce have good integration between projects to reduce shell script bootstrap cost. Earlier
version of YARN did not follow the conventional wisdom and it was hard to integrate with rest
of Hadoop, YARN community struggled on classpath issue for at least 2+ years and the time
to hype YARN framework had already passed. Given there is a high probability that we want
to make ozone as universal as possible for applications to integrate with us. It given us
more incentive to make the structure as flexible as possible. This is only a advice from my
own past scaring. There are no perfect solution, but the conventional wisdom usually have
endure test of time and save energy.
> {quote}
> I propose the following method:
>  # Start an independent discussion, let's don't mix this question with HDDS-1458 and
don't block it.
>  # Let's propose a more use friendly directory structure first
>  # Let's ask [~eyang] to definedĀ _technical_ problems regarding to the proposal.
> There are two different view here:
>  # Technical limitations which were mentioned earlier
>  # Theoretical questions (do we need to have the same structure for core Hadoop and Ozone
> I propose to start the discussion with the first one.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message