incubator-flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eran Kutner <e...@gigya.com>
Subject Collector going crazy...
Date Mon, 08 Aug 2011 14:41:52 GMT
Hi,
I have 3 collectors running, all with this configuration:
collectorSource(54001) | collector(600000) {
escapedFormatDfs("hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec")) }

Right now tall three are going crazy, opening multiple HDFS files per
second. All failing with this error: "OutputFormat instance can only write
to the same OutputStream".
This configuration used to work fine, for a few days, but I've seen this
wild behavior happen a few times before. Usually restarting the service was
enough to stop it. Now that doesn't seem to help.

The logs look like this:

11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
'WEB514 [INFO Sun Aug 07 09:51:43 EDT 2011] { City :  } { CookieID :  } {
Country : US } { Date : (long)-721216654471344504  (string) '??????F #??'
(double)-2.2850700827524182E260 } { ErrCode : 0 '' } { Event : 637534208 '&'
} { GigyaUID :  } { IP : 75.107.140.48 } { IPNum :
(long)5434591513138954240  (string) 'Kk???0' (double)2.110829276777375E55 }
{ Info :  } { PartnerData :  } { PartnerData_hash : (long)0  (string) ''
(double)0.0 } { PartnerID : (long)8768254487304339456  (string) 'y??? '
(double)1.3781410651737394E278 } { ProviderUID :  } { Referrer :
http://www.heavy.com/action/girls/2011/08/the-20-hottest-photos-of-jessica-jane-clement/}
{ ReqID : d3b9c9f27a1848919003e25268b90c34 } { RetryCount : 0 '' } {
RowType : GSEvent } { Server : WEB514 } { SiteAccountID :  } {
SiteAccountSNID : 0 '' } { State :  } { Status : 50331648 ' ' } { TotalTime
: 0 '' } { Type : js } { UCID : tSUUKaA13PGc62n9R9xn4PbqBwHxfeeuV15smIXotZY=
} { UUID :  } { UserAge : 48 } { UserGender : 0 '' } { Val : 0 '' } { dbTime
: 0 '' } { log :  } { memcacheTime : 0 '' } { provider : 0 '' } { request :
} { siteID : (long)8919959604146733056  (string) '{??? '
(double)1.9842753626727537E288 } { source : 1315925605 'None' } ' with
error: OutputFormat instance can only write to the same OutputStream
11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink 'escapedFormatDfs(
"hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
11/08/08 10:33:12 INFO rolling.RollSink: opening RollSink
'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
11/08/08 10:33:12 INFO debug.InsistentOpenDecorator: Opened MaskDecorator on
try 0
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
'WEB204 [INFO Mon Aug 08 10:13:54 EDT 2011] { City :  } { CookieID :  } {
Country : DE } { Date : (long)6591305623858826888  (string) '[y??????#??'
(double)4.440113505901932E132 } { IP : 93.231.131.246 } { IPNum :
(long)6766522057732653056  (string) ']??????' (double)2.2940434972655672E144
} { Referrer :  } { RowType : GSLoad } { Server : WEB204 } { State :  } {
UCID : Jyg2gzYokIn3id2wDfv71Q== } { isHTTPS : 48 } { partnerID :
(long)-7147478690450898944  (string) '??????' (double)-6.42864763300399E-170
} { siteID : (long)5554354511524397056  (string) 'M'
(double)2.162938607572601E63 } ' with error: OutputFormat instance can only
write to the same OutputStream
11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink 'escapedFormatDfs(
"hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs





-eran

Mime
View raw message