incubator-bigtop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcos Ortiz (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (BIGTOP-456) Consider splitting homedir between mapred and hdfs users?
Date Thu, 15 Mar 2012 23:56:37 GMT

    [ https://issues.apache.org/jira/browse/BIGTOP-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230719#comment-13230719
] 

Marcos Ortiz edited comment on BIGTOP-456 at 3/15/12 11:56 PM:
---------------------------------------------------------------


Yes, the /var/run/hadoop directory should be the right for it, although 
on the new versions of Fedora (15 or more), the default directory was 
changed to /run, so I don't know which is the best approach for it.

Fedora 15 Releases Notes
3. Changes for System Administrators
3.2.2 /run directory [1]

"Fedora 15 has a /run directory for storing runtime data. /run is 
now a tmpfs, and /var/run is bind mounted to it. /var/lock is 
as /var/run. Several programs including udev, dracut, mdadm, 
runtime data during early bootup before /var is mounted. However 
consensus between major distributions to shift to using /run instead. 
Fedora 15 is leading this change. Details including the benefits are 
explained here [2].

This change is compliant with the Filesystem Hierarchy Standard [3]
, which allows distributions to create new directories in the root hierarchy as 
long as there is careful consideration of the consequences. Co-author of 
the latest FHS specification has expressed support [4] for this change. Lennart Poettering

has filed a request [5] to update the FHS standard to include this change as well."

3.2.3 /var/run/  and /var/lock
ensure to recreate their own files/dirs on startup, and cannot rely that 
doing this at package installation will suffice. It is possible to use 
systemd's |tmpfiles.d| mechanism to recreate directories and files 
beneath /var/run and /var/lock on boot, if necessary. See 
(http://0pointer.de/public/systemd-man/tmpfiles.d.html) and the conf 
files in /etc/tmpfiles.d for examples of such configuration. Fedora 
packaging guidelines for tmpfiles.d is at 
http://fedoraproject.org/wiki/Packaging:Tmpfiles.d.

[1] http://docs.fedoraproject.org/en-US/Fedora/15/html/Release_Notes/sect-Release_Notes-Changes_for_SysAdmin.html
[2] http://lists.fedoraproject.org/pipermail/devel/2011-March/150031.html
[3] http://www.pathname.com/fhs/pub/fhs-2.3.html#THEROOTFILESYSTEM
[4] https://lwn.net/Articles/436177/
[5] http://bugs.freestandards.org/show_bug.cgi?id=718
                
      was (Author: mlortiz@uci.cu):
    

Yes, the /var/run/hadoop directory should be the right for it, although 
on the new versions of Fedora (15 or more), the default directory was 
changed to /run, so I don't know which is the best approach for it.

Fedora 15 Releases Notes
3. Changes for System Administrators
3.2.2 /run directory

"Fedora 15 has a |/run| directory for storing runtime data. |/run| is 
now a tmpfs, and |/var/run| is |bind| mounted to it. |/var/lock| is 
as |/var/run|. Several programs including |udev|, |dracut|, |mdadm|, 
runtime data during early bootup before |/var| is mounted. However 
consensus between major distributions to shift to using |/run| instead. 
Fedora 15 is leading this change. Details including the benefits are 
explained here 
<http://lists.fedoraproject.org/pipermail/devel/2011-March/150031.html>.
This change /is/ compliant with the Filesystem Hierarchy Standard 
<http://www.pathname.com/fhs/pub/fhs-2.3.html#THEROOTFILESYSTEM>, which 
allows distributions to create new directories in the root hierarchy as 
long as there is careful consideration of the consequences. Co-author of 
the latest FHS specification has expressed support 
<https://lwn.net/Articles/436177/> for this change. Lennart Poettering 
has filed a request <http://bugs.freestandards.org/show_bug.cgi?id=718> 
to update the FHS standard to include this change as well."

3.2.3 /var/run/  and /var/lock
ensure to recreate their own files/dirs on startup, and cannot rely that 
doing this at package installation will suffice. It is possible to use 
systemd's |tmpfiles.d| mechanism to recreate directories and files 
beneath |/var/run| and |/var/lock| on boot, if necessary. See 
(http://0pointer.de/public/systemd-man/tmpfiles.d.html) and the conf 
files in |/etc/tmpfiles.d| for examples of such configuration. Fedora 
packaging guidelines for |tmpfiles.d| is at 
http://fedoraproject.org/wiki/Packaging:Tmpfiles.d.


http://docs.fedoraproject.org/en-US/Fedora/15/html/Release_Notes/sect-Release_Notes-Changes_for_SysAdmin.html
Regards


-- 
Marcos Luis Ortíz Valmaseda (@marcosluis2186)
  Data Engineer at UCI
  http://marcosluis2186.posterous.com



10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

                  
> Consider splitting homedir between mapred and hdfs users?
> ---------------------------------------------------------
>
>                 Key: BIGTOP-456
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-456
>             Project: Bigtop
>          Issue Type: Improvement
>          Components: General
>    Affects Versions: 0.1.0
>         Environment: RPMs
>            Reporter: Harsh J
>            Priority: Minor
>
> Both "mapred" and "hdfs" users have the same home dir.
> A user reported having some problems with their config management system overwriting
the "mapred" user permissions of the PID directory (Which is also its homedir) with those
of the "hdfs" user (Same homedir as "mapred" user), which causes the tasktracker process to
fail to start, since it now cannot write to the PID dir.
> Although the config system can be fixed not to do that, if both users had separate home
dirs, this would not have been a problem, and the separation would have only been logical.
> I think after the username separation Hadoop has had in packaging terms, the homedir
split does make sense.
> Its just 1/0.22 versions of Hadoop and their packages that could be affected by this.
> Presently, for 0.23+, I think we have /var/run/hadoop/ for all things HDFS (Should we
rename?) and /var/run/yarn/ for all things MapReduce2 which makes sense and should be good
enough.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message