Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
Date: Sun, 26 Aug 2012 13:32:21 -0400
Message-ID: 
 <CANx3uAjYFzx=6OtUXtbvhbe4x96DQTr3OegM+SC8ibzuiN_4OQ@mail.gmail.com>
Subject: Is there a way to turn off MAPREDUCE-2415?
From: Koert Kuipers <koert@tresata.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=14dae93407f7715bf104c82e976d

--14dae93407f7715bf104c82e976d
Content-Type: text/plain; charset=ISO-8859-1

We have smaller nodes (4 to 6 disks), and we used to write logs to the same
disk as where the OS is. So if that disks goes then i don't really care
about tasktrackers failing. Also, the fact that logs were written to a
single partition meant that i could make sure they would not grow too large
in case someone had too verbose logging on a large job. With MAPREDUCE-2415
a job that does massive amount of logging can fill up all the
mapred.local.dir, which in our case are on the same partition as the hdfs
data dirs, so now faulty logging can fill up hdfs storage, which i really
don't like. Any ideas?

--14dae93407f7715bf104c82e976d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<font face=3D"courier new,monospace">We have smaller nodes (4 to 6 disks), =
and we used to write logs to the same disk as where the OS is. So if that d=
isks goes then i don&#39;t really care about tasktrackers failing. Also, th=
e fact that logs were written to a single partition meant that i could make=
 sure they would not grow too large in case someone had too verbose logging=
 on a large job. With MAPREDUCE-2415 a job that does massive amount of logg=
ing can fill up all the mapred.local.dir, which in our case are on the same=
 partition as the hdfs data dirs, so now faulty logging can fill up hdfs st=
orage, which i really don&#39;t like. Any ideas?<br>
<br><br></font>

--14dae93407f7715bf104c82e976d--