kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Andre Pearce (IG) (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-4497) log cleaner breaks on timeindex
Date Thu, 05 Jan 2017 12:34:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801230#comment-15801230
] 

Michael Andre Pearce (IG) edited comment on KAFKA-4497 at 1/5/17 12:34 PM:
---------------------------------------------------------------------------

[~becket_qin] we have managed to recover the log4j cleaner log from yesterdays, from the same
node we supplied the log segments. 

We have now re rolled back forward that testing environment back onto 0.10.1.1 and waiting
for it to re-occur. (bear in mind we had been running 6 weeks before we saw the initial issue
on 0.10.1.0 issue, whilst subsequently we have seen it re-occur much faster, we cannot tell
how long we will have to wait)


was (Author: michael.andre.pearce):
[~becket_qin] we have managed to recover the log4j cleaner log from yesterdays, from the same
node we supplied the log segments. 

We have now re rolled back forward that testing environment back onto 0.10.1.1 and waiting
for it to re-occur. (bear in mind we had been running 6 weeks before we saw the initial issue
on 0.10.1.0 issue, whilst subsequently we have seen it much faster, we cannot tell how long
we will have to wait)

> log cleaner breaks on timeindex
> -------------------------------
>
>                 Key: KAFKA-4497
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4497
>             Project: Kafka
>          Issue Type: Bug
>          Components: log
>    Affects Versions: 0.10.1.0
>         Environment: Debian Jessie, Oracle Java 8u92, kafka_2.11-0.10.1.0
>            Reporter: Robert Schumann
>            Assignee: Jiangjie Qin
>            Priority: Critical
>              Labels: reliability
>             Fix For: 0.10.1.1
>
>         Attachments: com_ig_trade_v1_order_event--demo--compacted-7.tar.bz2, vrtstokf005_4thJan
>
>
> _created from ML entry by request of [~ijuma]_
> Hi all,
> we are facing an issue with latest kafka 0.10.1 and the log cleaner thread with regards
to the timeindex files. From the log of the log-cleaner we see after startup that it tries
to cleanup a topic called xdc_listing-status-v2 [1]. The topic is setup with log compaction
[2] and the kafka cluster configuration has log.cleaner enabled [3]. Looking at the log and
the newly created file [4], the cleaner seems to refer to tombstones prior to epoch_time=0
- maybe because he finds messages, which don’t have a timestamp at all (?). All producers
and consumers are using 0.10.1 and the topics have been created completely new, so I’m not
sure, where this issue would come from. The original timeindex file [5] seems to show only
valid timestamps for the mentioned offsets. I would also like to mention that the issue happened
in two independent datacenters at the same time, so I would rather expect an application/producer
issue instead of random disk failures. We didn’t have the problem with 0.10.0 for around
half a year, it appeared shortly after the upgrade to 0.10.1.
> The puzzling message from the cleaner “cleaning prior to Fri Dec 02 16:35:50 CET 2016,
discarding tombstones prior to Thu Jan 01 01:00:00 CET 1970” also confuses me a bit. Does
that mean, it does not find any log segments which can be cleaned up or the last timestamp
of the last log segment is somehow broken/missing?
> I’m also a bit wondering, why the log cleaner thread stops completely after an error
with one topic. I would at least expect that it keeps on cleaning up other topics, but apparently
it doesn’t do that, e.g. it’s not even cleaning the __consumer_offsets anymore.
> Does anybody have the same issues or can explain, what’s going on? Thanks for any help
or suggestions.
> Cheers
> Robert
> [1]
> {noformat}
> [2016-12-06 12:49:17,885] INFO Starting the log cleaner (kafka.log.LogCleaner)
> [2016-12-06 12:49:17,895] INFO [kafka-log-cleaner-thread-0], Starting  (kafka.log.LogCleaner)
> [2016-12-06 12:49:17,947] INFO Cleaner 0: Beginning cleaning of log xdc_listing-status-v2-1.
(kafka.log.LogCleaner)
> [2016-12-06 12:49:17,948] INFO Cleaner 0: Building offset map for xdc_listing-status-v2-1...
(kafka.log.LogCleaner)
> [2016-12-06 12:49:17,989] INFO Cleaner 0: Building offset map for log xdc_listing-status-v2-1
for 1 segments in offset range [0, 194991). (kafka.log.LogCleaner)
> [2016-12-06 12:49:24,572] INFO Cleaner 0: Offset map for log xdc_listing-status-v2-1
complete. (kafka.log.LogCleaner)
> [2016-12-06 12:49:24,577] INFO Cleaner 0: Cleaning log xdc_listing-status-v2-1 (cleaning
prior to Fri Dec 02 16:35:50 CET 2016, discarding tombstones prior to Thu Jan 01 01:00:00
CET 1970)... (kafka.log.LogCleaner)
> [2016-12-06 12:49:24,580] INFO Cleaner 0: Cleaning segment 0 in log xdc_listing-status-v2-1
(largest timestamp Fri Dec 02 16:35:50 CET 2016) into 0, retaining deletes. (kafka.log.LogCleaner)
> [2016-12-06 12:49:24,968] ERROR [kafka-log-cleaner-thread-0], Error due to  (kafka.log.LogCleaner)
> kafka.common.InvalidOffsetException: Attempt to append an offset (-1) to slot 9 no larger
than the last offset appended (11832) to /var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned.
>         at kafka.log.TimeIndex$$anonfun$maybeAppend$1.apply$mcV$sp(TimeIndex.scala:117)
>         at kafka.log.TimeIndex$$anonfun$maybeAppend$1.apply(TimeIndex.scala:107)
>         at kafka.log.TimeIndex$$anonfun$maybeAppend$1.apply(TimeIndex.scala:107)
>         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:234)
>         at kafka.log.TimeIndex.maybeAppend(TimeIndex.scala:107)
>         at kafka.log.LogSegment.append(LogSegment.scala:106)
>         at kafka.log.Cleaner.cleanInto(LogCleaner.scala:518)
>         at kafka.log.Cleaner$$anonfun$cleanSegments$1.apply(LogCleaner.scala:404)
>         at kafka.log.Cleaner$$anonfun$cleanSegments$1.apply(LogCleaner.scala:400)
>         at scala.collection.immutable.List.foreach(List.scala:381)
>         at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:400)
>         at kafka.log.Cleaner$$anonfun$clean$4.apply(LogCleaner.scala:364)
>         at kafka.log.Cleaner$$anonfun$clean$4.apply(LogCleaner.scala:363)
>         at scala.collection.immutable.List.foreach(List.scala:381)
>         at kafka.log.Cleaner.clean(LogCleaner.scala:363)
>         at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:239)
>         at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:218)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
> [2016-12-06 12:49:24,971] INFO [kafka-log-cleaner-thread-0], Stopped  (kafka.log.LogCleaner)
> {noformat}
> [2]
> {noformat}
> $ kafka-configs-topics xdc_listing-status-v2 --describe
> Configs for topic 'xdc_listing-status-v2' are cleanup.policy=compact
> {noformat}
> [3]
> {noformat}
> broker.id=74
> listeners = PLAINTEXT://0.0.0.0:9092,TRACE://localhost:9093
> advertised.listeners = PLAINTEXT://broker2:9092
> num.network.threads=2
> num.io.threads=2
> inter.broker.protocol.version=0.10.1.0
> socket.send.buffer.bytes=1048576
> socket.receive.buffer.bytes=1048576
> socket.request.max.bytes=104857600
> log.dirs=/var/lib/kafka
> num.partitions=3
> default.replication.factor=2
> log.flush.interval.messages=10000
> log.flush.interval.ms=1000
> log.retention.hours=168
> log.segment.bytes=536870912
> log.cleanup.interval.mins=1
> zookeeper.connect=broker1:2181,broker2:2181,broker3:2181
> zookeeper.connection.timeout.ms=300000
> kafka.metrics.reporters=com.airbnb.kafka.KafkaStatsdMetricsReporter
> external.kafka.statsd.reporter.enabled=true
> external.kafka.statsd.host=localhost
> external.kafka.statsd.port=8125
> external.kafka.statsd.tag.enabled=false
> external.kafka.statsd.metrics.prefix=collectd.broker2
> kafka.metrics.polling.interval.secs=60
> external.kafka.statsd.dimension.enabled.meanRate=false
> external.kafka.statsd.dimension.enabled.rate15m=false
> external.kafka.statsd.dimension.enabled.min=false
> external.kafka.statsd.dimension.enabled.max=false
> external.kafka.statsd.dimension.enabled.median=false
> external.kafka.statsd.dimension.enabled.p75=false
> external.kafka.statsd.dimension.enabled.p98=false
> external.kafka.statsd.dimension.enabled.p999=false
> delete.topic.enable=true
> controlled.shutdown.enable=true
> auto.leader.rebalance.enable=true
> auto.create.topics.enable=false
> log.cleaner.enable=true
> offsets.topic.compression.codec=2
> offsets.topic.num.partitions=20
> {noformat}
> [4]
> {noformat}
> # ti=/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned; for
e in $(xxd -c 12 -p $ti); do echo -n "file:[$ti] raw:[$e] "; d="$(date -d @$(echo -n $((16#${e:0:16}))
| sed 's/.\{10\}/&./g'))"; echo -n "timestamp:[$d] "; echo "offset:[$((16#${e:16:24}))]";done
| head -n 20
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589c490ae1ffffffff] timestamp:[Fri Nov 25 17:19:42 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589c81f249ffffffff] timestamp:[Fri Nov 25 18:21:51 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589d4b4ea7ffffffff] timestamp:[Fri Nov 25 22:01:48 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589e04a783ffffffff] timestamp:[Sat Nov 26 01:24:14 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589ede5af6ffffffff] timestamp:[Sat Nov 26 05:22:02 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589f8ccc16ffffffff] timestamp:[Sat Nov 26 08:32:34 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000001589ffb82bcffffffff] timestamp:[Sat Nov 26 10:33:30 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[00000158a04fef7fffffffff] timestamp:[Sat Nov 26 12:05:43 CET 2016] offset:[4294967295]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[00000158a0b506eb00002e38] timestamp:[Sat Nov 26 13:56:08 CET 2016] offset:[11832]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex.cleaned]
raw:[000000000000000000000000] timestamp:[Thu Jan  1 01:00:00 CET 1970] offset:[0]
> {noformat}
> [5]
> {noformat}
> /var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex
> ...
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a06cbddc00002a21]
timestamp:[Sat Nov 26 12:37:10 CET 2016] offset:[10785]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a06cded500002a29]
timestamp:[Sat Nov 26 12:37:19 CET 2016] offset:[10793]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a06d25d400002a31]
timestamp:[Sat Nov 26 12:37:37 CET 2016] offset:[10801]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a06dc34700002a39]
timestamp:[Sat Nov 26 12:38:17 CET 2016] offset:[10809]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a06e3ccf00002a41]
timestamp:[Sat Nov 26 12:38:48 CET 2016] offset:[10817]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07051ae00002a4a]
timestamp:[Sat Nov 26 12:41:05 CET 2016] offset:[10826]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a071168100002a52]
timestamp:[Sat Nov 26 12:41:55 CET 2016] offset:[10834]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a071603400002a5a]
timestamp:[Sat Nov 26 12:42:14 CET 2016] offset:[10842]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07196c600002a62]
timestamp:[Sat Nov 26 12:42:28 CET 2016] offset:[10850]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a072531500002a6a]
timestamp:[Sat Nov 26 12:43:16 CET 2016] offset:[10858]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a072c5f700002a73]
timestamp:[Sat Nov 26 12:43:46 CET 2016] offset:[10867]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07340a500002a7b]
timestamp:[Sat Nov 26 12:44:17 CET 2016] offset:[10875]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0743e5a00002a85]
timestamp:[Sat Nov 26 12:45:22 CET 2016] offset:[10885]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a074d37b00002a8d]
timestamp:[Sat Nov 26 12:46:00 CET 2016] offset:[10893]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07551fe00002a95]
timestamp:[Sat Nov 26 12:46:33 CET 2016] offset:[10901]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a075e21900002a9e]
timestamp:[Sat Nov 26 12:47:09 CET 2016] offset:[10910]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a075faa900002aa6]
timestamp:[Sat Nov 26 12:47:16 CET 2016] offset:[10918]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07619f900002aae]
timestamp:[Sat Nov 26 12:47:24 CET 2016] offset:[10926]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a076447500002ab6]
timestamp:[Sat Nov 26 12:47:35 CET 2016] offset:[10934]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a076728000002abf]
timestamp:[Sat Nov 26 12:47:46 CET 2016] offset:[10943]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0768e7d00002ac8]
timestamp:[Sat Nov 26 12:47:54 CET 2016] offset:[10952]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a076af9300002ad1]
timestamp:[Sat Nov 26 12:48:02 CET 2016] offset:[10961]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a076d81500002ad9]
timestamp:[Sat Nov 26 12:48:12 CET 2016] offset:[10969]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0773f2200002ae1]
timestamp:[Sat Nov 26 12:48:39 CET 2016] offset:[10977]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a077babf00002ae9]
timestamp:[Sat Nov 26 12:49:10 CET 2016] offset:[10985]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0792cee00002af1]
timestamp:[Sat Nov 26 12:50:45 CET 2016] offset:[10993]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a079cb9b00002af9]
timestamp:[Sat Nov 26 12:51:26 CET 2016] offset:[11001]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07a28a800002b02]
timestamp:[Sat Nov 26 12:51:50 CET 2016] offset:[11010]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07a651500002b0a]
timestamp:[Sat Nov 26 12:52:05 CET 2016] offset:[11018]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07c0d0800002b14]
timestamp:[Sat Nov 26 12:53:54 CET 2016] offset:[11028]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07cc5cd00002b1c]
timestamp:[Sat Nov 26 12:54:41 CET 2016] offset:[11036]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07d807300002b25]
timestamp:[Sat Nov 26 12:55:29 CET 2016] offset:[11045]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07de02c00002b2d]
timestamp:[Sat Nov 26 12:55:53 CET 2016] offset:[11053]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07e505c00002b35]
timestamp:[Sat Nov 26 12:56:22 CET 2016] offset:[11061]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07e6c6e00002b3d]
timestamp:[Sat Nov 26 12:56:29 CET 2016] offset:[11069]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07e8a2600002b45]
timestamp:[Sat Nov 26 12:56:37 CET 2016] offset:[11077]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07ed26b00002b4d]
timestamp:[Sat Nov 26 12:56:55 CET 2016] offset:[11085]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a07f729600002b55]
timestamp:[Sat Nov 26 12:57:36 CET 2016] offset:[11093]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a080278400002b5e]
timestamp:[Sat Nov 26 12:58:23 CET 2016] offset:[11102]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a08168e300002b67]
timestamp:[Sat Nov 26 12:59:45 CET 2016] offset:[11111]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a08257c900002b6f]
timestamp:[Sat Nov 26 13:00:46 CET 2016] offset:[11119]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a083f4b900002b77]
timestamp:[Sat Nov 26 13:02:32 CET 2016] offset:[11127]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a084d01c00002b7f]
timestamp:[Sat Nov 26 13:03:28 CET 2016] offset:[11135]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a086450500002b87]
timestamp:[Sat Nov 26 13:05:03 CET 2016] offset:[11143]
> ...
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09d4a7700002cff]
timestamp:[Sat Nov 26 13:30:12 CET 2016] offset:[11519]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09d783100002d07]
timestamp:[Sat Nov 26 13:30:24 CET 2016] offset:[11527]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09da2a100002d0f]
timestamp:[Sat Nov 26 13:30:35 CET 2016] offset:[11535]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09dcdf100002d17]
timestamp:[Sat Nov 26 13:30:46 CET 2016] offset:[11543]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09dfdb700002d1f]
timestamp:[Sat Nov 26 13:30:58 CET 2016] offset:[11551]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09e311700002d28]
timestamp:[Sat Nov 26 13:31:11 CET 2016] offset:[11560]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09eccaa00002d31]
timestamp:[Sat Nov 26 13:31:51 CET 2016] offset:[11569]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a09f542300002d3a]
timestamp:[Sat Nov 26 13:32:26 CET 2016] offset:[11578]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a026c100002d43]
timestamp:[Sat Nov 26 13:33:20 CET 2016] offset:[11587]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a0d8c900002d4b]
timestamp:[Sat Nov 26 13:34:05 CET 2016] offset:[11595]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a163d400002d53]
timestamp:[Sat Nov 26 13:34:41 CET 2016] offset:[11603]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a310fe00002d5c]
timestamp:[Sat Nov 26 13:36:31 CET 2016] offset:[11612]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a3b0a300002d65]
timestamp:[Sat Nov 26 13:37:11 CET 2016] offset:[11621]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a3cdf100002d6d]
timestamp:[Sat Nov 26 13:37:19 CET 2016] offset:[11629]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a42ba200002d75]
timestamp:[Sat Nov 26 13:37:43 CET 2016] offset:[11637]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a4864900002d7e]
timestamp:[Sat Nov 26 13:38:06 CET 2016] offset:[11646]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a500e700002d86]
timestamp:[Sat Nov 26 13:38:38 CET 2016] offset:[11654]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a5a38200002d8e]
timestamp:[Sat Nov 26 13:39:19 CET 2016] offset:[11662]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a5f5a400002d96]
timestamp:[Sat Nov 26 13:39:40 CET 2016] offset:[11670]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a6e30400002d9e]
timestamp:[Sat Nov 26 13:40:41 CET 2016] offset:[11678]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0a7e6b600002da7]
timestamp:[Sat Nov 26 13:41:47 CET 2016] offset:[11687]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0aa276300002db1]
timestamp:[Sat Nov 26 13:44:15 CET 2016] offset:[11697]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0ab040c00002dbb]
timestamp:[Sat Nov 26 13:45:12 CET 2016] offset:[11707]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0ab6d2c00002dc3]
timestamp:[Sat Nov 26 13:45:38 CET 2016] offset:[11715]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0aba59e00002dcb]
timestamp:[Sat Nov 26 13:45:53 CET 2016] offset:[11723]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0acb9b200002dd4]
timestamp:[Sat Nov 26 13:47:04 CET 2016] offset:[11732]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0ae188e00002ddc]
timestamp:[Sat Nov 26 13:48:33 CET 2016] offset:[11740]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0ae682b00002de4]
timestamp:[Sat Nov 26 13:48:54 CET 2016] offset:[11748]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0ae7a9b00002ded]
timestamp:[Sat Nov 26 13:48:59 CET 2016] offset:[11757]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0ae9bed00002df5]
timestamp:[Sat Nov 26 13:49:07 CET 2016] offset:[11765]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0aeb82200002dfd]
timestamp:[Sat Nov 26 13:49:14 CET 2016] offset:[11773]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0aed16a00002e05]
timestamp:[Sat Nov 26 13:49:21 CET 2016] offset:[11781]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0aedd8d00002e0d]
timestamp:[Sat Nov 26 13:49:24 CET 2016] offset:[11789]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0aff17200002e16]
timestamp:[Sat Nov 26 13:50:34 CET 2016] offset:[11798]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0b0d65e00002e1e]
timestamp:[Sat Nov 26 13:51:33 CET 2016] offset:[11806]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0b1e31f00002e26]
timestamp:[Sat Nov 26 13:52:42 CET 2016] offset:[11814]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0b3736300002e2e]
timestamp:[Sat Nov 26 13:54:24 CET 2016] offset:[11822]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0b4648a00002e36]
timestamp:[Sat Nov 26 13:55:26 CET 2016] offset:[11830]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0b7ad3d00002e40]
timestamp:[Sat Nov 26 13:59:01 CET 2016] offset:[11840]
> file:[/var/lib/kafka/xdc_listing-status-v2-1/00000000000000000000.timeindex] raw:[00000158a0b7efe700002e49]
timestamp:[Sat Nov 26 13:59:18 CET 2016] offset:[11849]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message