apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganelin, Ilya" <Ilya.Gane...@capitalone.com>
Subject Strange pauses with HDFS FileOutput operator
Date Tue, 14 Mar 2017 05:34:01 GMT
Hi, all – I’m seeing strange behavior in my app. I’m observing periodic dropouts of “tuples
processed” as shown in the picture below. I theorized that this has to do with the flushSize
for the AbstractFileOutputOperator I’m extending but that doesn’t seem to be the case.
The other option is that this has to do with the compression stream I’ve added, has anyone
seen something similar before?
https://gist.github.com/ilganeli/1326723b67d2f1c571e059a3593e02ab

[cid:image001.png@01D29C49.E9076F90]


- Ilya Ganelin
[id:image001.png@01D1F7A4.F3D42980]
________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates and may only be used solely in performance of work or services for Capital
One. The information transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended recipient, you
are hereby notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is strictly prohibited.
If you have received this communication in error, please contact the sender and delete the
material from your computer.
Mime
View raw message