kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mayuresh Gharat (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1763) validate_index_log in system tests runs remotely but uses local paths
Date Thu, 13 Nov 2014 21:08:34 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211289#comment-14211289
] 

Mayuresh Gharat commented on KAFKA-1763:
----------------------------------------

Traceback (most recent call last):
 File 
"/mnt/u001/kafka_replication_system_test/system_test/replication_testsuite/
replica_basic_test.py", line 434, in runTest

kafka_system_test_utils.validate_simple_consumer_data_matched_across_replic
as(self.systemTestEnv, self.testcaseEnv)
 File 
"/mnt/u001/kafka_replication_system_test/system_test/utils/kafka_system_tes
t_utils.py", line 2223, in
validate_simple_consumer_data_matched_across_replicas
   replicaIdxMsgIdList[replicaIdx - 1][topicPartition] = consumerMsgIdList
IndexError: list index out of range

> validate_index_log in system tests runs remotely but uses local paths
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-1763
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1763
>             Project: Kafka
>          Issue Type: Bug
>          Components: system tests
>    Affects Versions: 0.8.1.1
>            Reporter: Ewen Cheslack-Postava
>            Assignee: Ewen Cheslack-Postava
>             Fix For: 0.8.3
>
>         Attachments: KAFKA-1763.patch
>
>
> validate_index_log is the only validation step in the system tests that needs to execute
a Kafka binary and it's currently doing so remotely, like the rest of the test binaries. However,
this is probably incorrect since it looks like logs are synced back to the driver host and
in other cases are operated on locally. It looks like validate_index_log mixes up local/remote
paths, causing an exception in DumpLogSegments:
> {quote}
> 2014-11-10 12:09:57,665 - DEBUG - executing command [ssh vagrant@worker1 -o 'HostName
127.0.0.1' -o 'Port 2222' -o 'UserKnownHostsFile /dev/null' -o 'StrictHostKeyChecking no'
-o 'PasswordAuthentication no' -o 'IdentityFile /Users/ewencp/.vagrant.d/insecure_private_key'
-o 'IdentitiesOnly yes' -o 'LogLevel FATAL'  '/opt/kafka/bin/kafka-run-class.sh kafka.tools.DumpLogSegments
 --file /Users/ewencp/kafka.git/system_test/replication_testsuite/testcase_0008/logs/broker-3/kafka_server_3_logs/test_1-2/00000000000000001294.index
--verify-index-only 2>&1'] (system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - Dumping /Users/ewencp/kafka.git/system_test/replication_testsuite/testcase_0008/logs/broker-3/kafka_server_3_logs/test_1-2/00000000000000001294.index
(kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - Exception in thread "main" java.io.FileNotFoundException:
/Users/ewencp/kafka.git/system_test/replication_testsuite/testcase_0008/logs/broker-3/kafka_server_3_logs/test_1-2/00000000000000001294.log
(No such file or directory) (kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - 	at java.io.FileInputStream.open(Native Method) (kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - 	at java.io.FileInputStream.<init>(FileInputStream.java:146)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - 	at kafka.utils.Utils$.openChannel(Utils.scala:162)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - 	at kafka.log.FileMessageSet.<init>(FileMessageSet.scala:74)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - 	at kafka.tools.DumpLogSegments$.kafka$tools$DumpLogSegments$$dumpIndex(DumpLogSegments.scala:108)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,673 - DEBUG - 	at kafka.tools.DumpLogSegments$$anonfun$main$1.apply(DumpLogSegments.scala:80)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,674 - DEBUG - 	at kafka.tools.DumpLogSegments$$anonfun$main$1.apply(DumpLogSegments.scala:73)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,674 - DEBUG - 	at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,674 - DEBUG - 	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:105)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,674 - DEBUG - 	at kafka.tools.DumpLogSegments$.main(DumpLogSegments.scala:73)
(kafka_system_test_utils)
> 2014-11-10 12:09:58,674 - DEBUG - 	at kafka.tools.DumpLogSegments.main(DumpLogSegments.scala)
(kafka_system_test_utils)
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message