Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1410A1068C for ; Wed, 24 Jul 2013 06:53:26 +0000 (UTC) Received: (qmail 95919 invoked by uid 500); 24 Jul 2013 06:53:19 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 95529 invoked by uid 500); 24 Jul 2013 06:53:18 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 95522 invoked by uid 99); 24 Jul 2013 06:53:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jul 2013 06:53:18 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of Ajay.Srivastava@guavus.com designates 204.232.241.167 as permitted sender) Received: from [204.232.241.167] (HELO mx1.guavus.com) (204.232.241.167) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jul 2013 06:53:11 +0000 Received: from MX3.guavus.com ([fe80::914d:92ba:eba4:253f]) by mx1.guavus.com ([204.232.241.167]) with mapi id 14.01.0438.000; Tue, 23 Jul 2013 23:52:30 -0700 From: Ajay Srivastava To: "" Subject: Re: Only log.index Thread-Topic: Only log.index Thread-Index: AQHOh8JmnDDYobqhQEiKs6fqesL5n5lzxjAAgAAEbwCAAAx8gIAAA3EA Date: Wed, 24 Jul 2013 06:52:28 +0000 Message-ID: References: <4BC2848E-E41F-4235-A437-CC3515110D42@guavus.com> <20FC64C0-265D-429B-B3BC-5929D9649DB2@apache.org> In-Reply-To: <20FC64C0-265D-429B-B3BC-5929D9649DB2@apache.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [61.12.3.120] Content-Type: multipart/alternative; boundary="_000_BC863497AF634047B080201661394281guavuscom_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_BC863497AF634047B080201661394281guavuscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yes. That explains it and confirms my guess too :-) stderr:156 0 syslog:995 166247 What are these numbers ? Byte offset in corresponding files from where logs= of this task starts. Regards, Ajay Srivastava On 24-Jul-2013, at 12:10 PM, Vinod Kumar Vavilapalli wrote: Ah, I should've guessed that. You seem to have JVM reuse enabled. Even if J= VMs are reused, all the tasks write to the same files as they share the JVM= . They only have different index files. The same thing happens for what we = call the TaskCleanup tasks which are launched for failing/killed tasks. Thanks, +Vinod On Jul 23, 2013, at 10:55 PM, Ajay Srivastava wrote: Hi Vinod, Thanks. It seems that something else is going on - Here is the content of log.index - ajay-srivastava:userlogs ajay.srivastava$ cat job_201307222115_0188/attempt= _201307222115_0188_r_000000_0/log.index LOG_DIR:/opt/hadoop/bin/../logs/userlogs/job_201307222115_0188/attempt_2013= 07222115_0188_r_000008_0 stdout:0 0 stderr:156 0 syslog:995 166247 Looks like that the log.index is pointing to another attempt directory. Is it doing some kind of optimization ? What is purpose of log.index ? Regards, Ajay Srivastava On 24-Jul-2013, at 11:09 AM, Vinod Kumar Vavilapalli wrote: It could either mean that all those task-attempts are crashing before the p= rocess itself is getting spawned (check TT logs) or those logs are getting = deleted after the fact. Suspect the earlier. Thanks, +Vinod On Jul 23, 2013, at 9:33 AM, Ajay Srivastava wrote: Hi, I see that most of the tasks have only log.index created in /opt/hadoop/log= s/userlogs/jobId/task_attempt directory. When does this happen ? Is there a config setting for this OR this is a bug ? Regards, Ajay Srivastava --_000_BC863497AF634047B080201661394281guavuscom_ Content-Type: text/html; charset="us-ascii" Content-ID: <532737926645F144AEBDCBA659D8ACF9@guavus.com> Content-Transfer-Encoding: quoted-printable
Yes. That explains it and confirms my guess too :-)

stderr:156 0
syslog:995 166247

What are these numbers ? Byte offset in corresponding files from where= logs of this task starts.



Regards,
Ajay Srivastava


On 24-Jul-2013, at 12:10 PM, Vinod Kumar Vavilapalli wrote:


Ah, I should've guessed that. You seem to have JVM reuse enabled. Even= if JVMs are reused, all the tasks write to the same files as they share th= e JVM. They only have different index files. The same thing happens for wha= t we call the TaskCleanup tasks which are launched for failing/killed tasks.

Thanks,
+Vinod

On Jul 23, 2013, at 10:55 PM, Ajay Srivastava wrote:

Hi Vinod,

Thanks. It seems that something else is going on -

Here is the content of log.index -

ajay-srivastava:userlogs ajay.srivastava$ cat job_201307222115_0188/attempt= _201307222115_0188_r_000000_0/log.index
LOG_DIR:/opt/hadoop/bin/../logs/userlogs/job_201307222115_0188/attempt_2013= 07222115_0188_r_000008_0
stdout:0 0
stderr:156 0
syslog:995 166247

Looks like that the log.index is pointing to another attempt directory.
Is it doing some kind of optimization ? What is purpose of log.index ?


Regards,
Ajay Srivastava


On 24-Jul-2013, at 11:09 AM, Vinod Kumar Vavilapalli wrote:


It could either mean that all those task-attempts= are crashing before the process itself is getting spawned (check TT logs) = or those logs are getting deleted after the fact. Suspect the earlier.

Thanks,
+Vinod

On Jul 23, 2013, at 9:33 AM, Ajay Srivastava wrot= e:

Hi,

I see that most of the tasks have only log.index = created in /opt/hadoop/logs/userlogs/jobId/task_attempt directory.
When does this happen ?
Is there a config setting for this OR this is a b= ug ?


Regards,
Ajay Srivastava




--_000_BC863497AF634047B080201661394281guavuscom_--