hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject understanding hadoop.tmp.dir
Date Fri, 12 May 2017 09:51:55 GMT

I've been looking for a location on HDFS for temp data as part of HADOOP-13786, which has
had me looking at hadoop.tmp.dir

  1.  There's no actual java constant for it
  2.  We don't spell out that it is in the local FS only...there's a risk that it is picked
up and used in the cluster path, which is wrong, especially if a config changes that temp
  3.  And as Milind & AW have noted in the past, it's a single dir, not a list of directories

Putting aside this confusion: we don't have any equivalent for the cluster fs, like "hdfs.tmp.dir"

what should I be using here? And if there isn't a good answer other than "use hadoop.tmp.dir
and hope the permissions are consistent", maybe its time to add one —even if the default
value for it is just hadoop.tmp.dir

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message