Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0898C200C40 for ; Thu, 23 Mar 2017 09:50:49 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 07158160B92; Thu, 23 Mar 2017 08:50:49 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EFDC0160B75 for ; Thu, 23 Mar 2017 09:50:47 +0100 (CET) Received: (qmail 76214 invoked by uid 500); 23 Mar 2017 08:50:47 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 76205 invoked by uid 99); 23 Mar 2017 08:50:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Mar 2017 08:50:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id BC093CC22E for ; Thu, 23 Mar 2017 08:50:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.349 X-Spam-Level: X-Spam-Status: No, score=-99.349 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id HEn1PlMyqLyn for ; Thu, 23 Mar 2017 08:50:45 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 4132A5FB40 for ; Thu, 23 Mar 2017 08:50:45 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 0B053E04A8 for ; Thu, 23 Mar 2017 08:50:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id B09F621D9A for ; Thu, 23 Mar 2017 08:50:41 +0000 (UTC) Date: Thu, 23 Mar 2017 08:50:41 +0000 (UTC) From: "Mike Lambert (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (BEAM-1791) Errors are truncated MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 23 Mar 2017 08:50:49 -0000 [ https://issues.apache.org/jira/browse/BEAM-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Lambert updated BEAM-1791: ------------------------------- Summary: Errors are truncated (was: Logs are truncated) > Errors are truncated > -------------------- > > Key: BEAM-1791 > URL: https://issues.apache.org/jira/browse/BEAM-1791 > Project: Beam > Issue Type: Bug > Components: sdk-py > Reporter: Mike Lambert > Assignee: Ahmet Altay > Priority: Minor > > When I run my code in the Google Cloud Dataflow, I am getting some tracebacks in the the "Errors" view, but they are truncated. (Not sure if this is a bug against this Jira component, or should be filed somewhere else...) > {noformat} > (cee7eb663424a856): Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 578, in do_work > work_executor.execute() > File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 165, in execute > op.start() > File "dataflow_worker/shuffle_operations.py", line 49, in dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start (dataflow_worker/shuffle_operations.c:3442) > def start(self): > File "dataflow_worker/shuffle_operations.py", line 50, in dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start (dataflow_worker/shuffle_operations.c:3346) > with self.scoped_start_state: > File "dataflow_worker/shuffle_operations.py", line 63, in dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start (dataflow_worker/shuffle_operations.c:3261) > with self.shuffle_source.reader() as reader: > File "dataflow_worker/shuffle_operations.py", line 67, in dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start (dataflow_worker/shuffle_operations.c:3205) > self.output(windowed_value) > File "dataflow_worker/operations.py", line 139, in dataflow_worker.operations.Operation.output (dataflow_worker/operations.c:5717) > cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_value) > File "dataflow_worker/shuffle_operations.py", line 220, in dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.process (dataflow_worker/shuffle_operations.c:8094) > self.output(wvalue.with_value((k, wvalue.value))) > File "dataflow_worker/operations.py", line 139, in dataflow_worker.operations.Operation.output (dataflow_worker/operations.c:5717) > cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_value) > File "dataflow_worker/operations.py", line 395, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13546) > with self.scoped_process_state: > File "dataflow_worker/operations.py", line 396, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13500) > self.dofn_receiver.receive(o) > File "apache_beam/runners/common.py", line 195, in apache_beam.runners.common.DoFnRunner.receive (apache_beam/runners/common.c:5142) > self.process(windowed_value) > File "apache_beam/runners/common.py", line 267, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7201) > self.reraise_augmented(exn) > File "apache_beam/runners/common.py", line 281, in apache_beam.runners.common.DoFnRunner.reraise_augmented (apache_beam/runners/common.c:7609) > raise > File "apache_beam/runners/common.py", line 263, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7090) > self._dofn_simple_invoker(element) > File "apache_beam/runners/common.py", line 198, in apache_beam.runners.common.DoFnRunner._dofn_simple_invoker (apache_beam/runners/common.c:5288) > self._process_outputs(element, self.dofn_process(element.value)) > File "apache_beam/runners/common.py", line 326, in apache_beam.runners.common.DoFnRunner._process_outputs (apache_beam/runners/common.c:8563) > self.main_receivers.receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_value) > File "dataflow_worker/operations.py", line 395, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13546) > with self.scoped_process_state: > File "dataflow_worker/operations.py", line 396, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13500) > self.dofn_receiver.receive(o) > File "apache_beam/runners/common.py", line 195, in apache_beam.runners.common.DoFnRunner.receive (apache_beam/runners/common.c:5142) > self.process(windowed_value) > File "apache_beam/runners/common.py", line 267, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7201) > self.reraise_augmented(exn) > File "apache_beam/runners/common.py", line 281, in apache_beam.runners.common.DoFnRunner.reraise_augmented (apache_beam/runners/common.c:7609) > raise > File "apache_beam/runners/common.py", line 263, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7090) > self._dofn_simple_invoker(element) > File "apache_beam/runners/common.py", line 198, in apache_beam.runners.common.DoFnRunner._dofn_simple_invoker (apache_beam/runners/common.c:5288) > self._process_outputs(element, self.dofn_process(element.value)) > File "apache_beam/runners/common.py", line 326, in apache_beam.runners.common.DoFnRunner._process_outputs (apache_beam/runners/common.c:8563) > self.main_receivers.receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_value) > File "dataflow_worker/operations.py", line 395, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13546) > with self.scoped_process_state: > File "dataflow_worker/operations.py", line 396, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13500) > self.dofn_receiver.receive(o) > File "apache_beam/runners/common.py", line 195, in apache_beam.runners.common.DoFnRunner.receive (apache_beam/runners/common.c:5142) > self.process(windowed_value) > File "apache_beam/runners/common.py", line 267, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7201) > self.reraise_augmented(exn) > File "apache_beam/runners/common.py", line 281, in apache_beam.runners.common.DoFnRunner.reraise_augmented (apache_beam/runners/common.c:7609) > raise > File "apache_beam/runners/common.py", line 263, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7090) > self._dofn_simple_invoker(element) > File "apache_beam/runners/common.py", line 198, in apache_beam.runners.common.DoFnRunner._dofn_simple_invoker (apache_beam/runners/common.c:5288) > self._process_outputs(element, self.dofn_process(element.value)) > File "apache_beam/runners/common.py", line 326, in apache_beam.runners.common.DoFnRunner._process_outputs (apache_beam/runners/common.c:8563) > self.main_receivers.receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_value) > File "dataflow_worker/operations.py", line 395, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13546) > with self.scoped_process_state: > File "dataflow_worker/operations.py", line 396, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13500) > self.dofn_receiver.receive(o) > File "apache_beam/runners/common.py", line 195, in apache_beam.runners.common.DoFnRunner.receive (apache_beam/runners/common.c:5142) > self.process(windowed_value) > File "apache_beam/runners/common.py", line 267, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7201) > self.reraise_augmented(exn) > File "apache_beam/runners/common.py", line 281, in apache_beam.runners.common.DoFnRunner.reraise_augmented (apache_beam/runners/common.c:7609) > raise > File "apache_beam/runners/common.py", line 263, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7090) > self._dofn_simple_invoker(element) > File "apache_beam/runners/common.py", line 198, in apache_beam.runners.common.DoFnRunner._dofn_simple_invoker (apache_beam/runners/common.c:5288) > self._process_outputs(element, self.dofn_process(element.value)) > File "apache_beam/runners/common.py", line 326, in apache_beam.runners.common.DoFnRunner._process_outputs (apache_beam/runners/common.c:8563) > self.main_receivers.receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_value) > File "dataflow_worker/operations.py", line 395, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13546) > with self.scoped_process_state: > File "dataflow_worker/operations.py", line 396, in dataflow_worker.operations.DoOperation.process (dataflow_worker/operations.c:13500) > self.dofn_receiver.receive(o) > File "apache_beam/runners/common.py", line 195, in apache_beam.runners.common.DoFnRunner.receive (apache_beam/runners/common.c:5142) > self.process(windowed_value) > File "apache_beam/runners/common.py", line 267, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7201) > self.reraise_augmented(exn) > File "apache_beam/runners/common.py", line 281, in apache_beam.runners.common.DoFnRunner.reraise_augmented (apache_beam/runners/common.c:7609) > raise > File "apache_beam/runners/common.py", line 263, in apache_beam.runners.common.DoFnRunner.process (apache_beam/runners/common.c:7090) > self._dofn_simple_invoker(element) > File "apache_beam/runners/common.py", line 198, in apache_beam.runners.common.DoFnRunner._dofn_simple_invoker (apache_beam/runners/common.c:5288) > self._process_outputs(element, self.dofn_process(element.value)) > File "apache_beam/runners/common.py", line 326, in apache_beam.runners.common.DoFnRunner._process_outputs (apache_beam/runners/common.c:8563) > self.main_receivers.receive(windowed_value) > File "dataflow_worker/operations.py", line 72, in dataflow_worker.operations.ConsumerSet.receive (dataflow_worker/operations.c:3421) > cython.cast(Operation, consumer).process(windowed_valu > {noformat} > Notice how it is truncated at the very end...after around 10220 characters, and it hasn't even gotten to my actual code's stacktrace yet! > If I open the logs panel and go look at the logs, I'm luckily able to see the full thing...including the very last line that is pointing at my code, as well as the actual exception class/message itself. -- This message was sent by Atlassian JIRA (v6.3.15#6346)