From users-return-7892-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Thu Feb 1 07:30:14 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 92E7F180652 for ; Thu, 1 Feb 2018 07:30:14 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 82E5D160C44; Thu, 1 Feb 2018 06:30:14 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CB9B1160C26 for ; Thu, 1 Feb 2018 07:30:13 +0100 (CET) Received: (qmail 44578 invoked by uid 500); 1 Feb 2018 06:30:12 -0000 Mailing-List: contact users-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@nifi.apache.org Delivered-To: mailing list users@nifi.apache.org Received: (qmail 44568 invoked by uid 99); 1 Feb 2018 06:30:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Feb 2018 06:30:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 48DBA198C30 for ; Thu, 1 Feb 2018 06:30:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1 X-Spam-Level: * X-Spam-Status: No, score=1 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id w7Mk7Ilc-Ny5 for ; Thu, 1 Feb 2018 06:30:11 +0000 (UTC) Received: from secure.my2ndhead.com (static.183.104.4.46.clients.your-server.de [46.4.104.183]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 398E05F296 for ; Thu, 1 Feb 2018 06:30:11 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by secure.my2ndhead.com (Postfix) with ESMTP id 607852280061 for ; Thu, 1 Feb 2018 07:30:04 +0100 (CET) Received: from secure.my2ndhead.com ([127.0.0.1]) by localhost (secure.my2ndhead.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id lGXuWbPtztHJ for ; Thu, 1 Feb 2018 07:30:03 +0100 (CET) Received: from localhost (localhost.localdomain [127.0.0.1]) by secure.my2ndhead.com (Postfix) with ESMTP id 69F1E2280069 for ; Thu, 1 Feb 2018 07:30:03 +0100 (CET) X-Virus-Scanned: amavisd-new at my2ndhead.com Received: from secure.my2ndhead.com ([127.0.0.1]) by localhost (secure.my2ndhead.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Y9wu-eSb1X-x for ; Thu, 1 Feb 2018 07:30:03 +0100 (CET) Received: from [192.168.178.100] (212-51-128-150.fiber7.init7.net [212.51.128.150]) by secure.my2ndhead.com (Postfix) with ESMTPSA id 133822280061 for ; Thu, 1 Feb 2018 07:30:03 +0100 (CET) Subject: Re: PublishKafka setting record timestamp To: users@nifi.apache.org References: <808f69e7-5a4f-1d89-3775-5622e1eac87b@my2ndhead.com> From: Mika Borner Message-ID: <53f41b79-2002-3a74-3a2a-060594dc7b25@my2ndhead.com> Date: Thu, 1 Feb 2018 07:30:02 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US Thanks Bryan For PublishKafka, couldn't this be done by using an attribute that contains the timestamp? For PublishKafkaRecord this sounds reasonable. I guess, I will open an enhancement request then. Mika> On 01/30/2018 02:58 PM, Bryan Bende wrote: > Hello, > > The timestamp in Kafka is separate from the headers, currently there > isn't a way to specify the timestamp from NiFi. > > For PublishKafkaRecord, I could see having an option to take the value > of a specified field from each record and make that the timestamp, > assuming it can be converted to a long. > > For PublishKafka, the processor doesn't have any knowledge about the > actual data, so the only thing that could be done here is to set the > same timestamp for every piece of data from the given flow file, which > doesn't seem as helpful. > > -Bryan > > > On Mon, Jan 29, 2018 at 5:37 AM, Mika Borner wrote: >> Hi, >> >> Is it possible to set the producer record timestamp within the >> PublishKafka_1_0 / PublishKafkaRecord_1_0 processor? >> >> I tried to use the "Attributes to Send as Headers" option with a timestamp >> attribute, but this did not work. Not sure if the timestamp producer >> record's timestamp is in the headers. >> >> Appreciate any help. >> >> Thanks, >> >> Mika>