kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <joe.st...@stealth.ly>
Subject Re: Format of Kafka storage on disk
Date Sat, 04 Jan 2014 01:23:09 GMT
The DumpLogSegments should do that for you

bin/kafka-run-class.sh kafka.tools.DumpLogSegments

Option                                  Description

------                                  -----------

--deep-iteration                        if set, uses deep instead of

--files <file1, file2, ...>             REQUIRED: The comma separated list
                                          data and index log files to be
--max-message-size <Integer: size>      Size of largest message. (default:


--print-data-log                        if set, printing the messages
                                          when dumping data logs

--verify-index-only                     if set, just verify the index log

                                          without printing its content

or use the code as entry point for whatever you want to-do :)

 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>

On Fri, Jan 3, 2014 at 5:10 PM, Subbu Srinivasan <ssriniva123@gmail.com>wrote:

> Is there any place where I can know about the internal structure of
> the log file where kafka stores the data. A topic has a .index and a .log
> file.
> I want to read the entire log file and parse the contents out.
> Thanks
> Subbu

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message