From dev-return-16818-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Tue Mar 27 08:35:58 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id BAF0A18064E for ; Tue, 27 Mar 2018 08:35:57 +0200 (CEST) Received: (qmail 55307 invoked by uid 500); 27 Mar 2018 06:35:56 -0000 Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list dev@nifi.apache.org Received: (qmail 55294 invoked by uid 99); 27 Mar 2018 06:35:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Mar 2018 06:35:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 443CE1A074A for ; Tue, 27 Mar 2018 06:35:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.129 X-Spam-Level: ** X-Spam-Status: No, score=2.129 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id zNEjmKz16x5g for ; Tue, 27 Mar 2018 06:35:53 +0000 (UTC) Received: from mail-lf0-f52.google.com (mail-lf0-f52.google.com [209.85.215.52]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id DF1E95F5B2 for ; Tue, 27 Mar 2018 06:35:52 +0000 (UTC) Received: by mail-lf0-f52.google.com with SMTP id z143-v6so31639139lff.3 for ; Mon, 26 Mar 2018 23:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=IA7r+6gG1Q2HjvlNJRWiNH/uEZAOiDxtX2JZyI1o8Rc=; b=jjKtJyFI0U8CNxqMKCf6ZI0/2OGhcR25Wa06wV+BNRpIGhFvwFokB694hmEnJSOzZw qgh+z057dgQcQG/4FNoCnKm79vxNcRhvFokUmL//16ifn41OSS7jqmreg7QzKlNwzpC8 NV7gR/7W6eVR2uLVcmLV8RWZ8FcMSP+r5QKQfoYOu5Rl4rZtl10iOJSpwVQ/sOKV/F+i R13lrzmgohGaKD+AXDMsD/C33qsDs7u4cKjaQpV6i9pfoGZZNsmJxTcwDqhkfL2WrvRE X+V0Z1mOCaxfUR+sr/vhoVBXAmsN7FlEgjI7jnMtN2i4VldCeo8JmuK6+8dqsWlWqcTn TAMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=IA7r+6gG1Q2HjvlNJRWiNH/uEZAOiDxtX2JZyI1o8Rc=; b=hO5qr2FXH34MSsPvXI+mD0SvcfL46H2iQLEhxvI2yYVX0rcEf7z0bmq0aYcmutQfYp OSHY/yCobz0NOH1OBxDHMRpu3R0fB29DuOdpoYsiseAbYfcyzX5ZTcrl9jJI+oBjbuli 5QQ7kfFUw4Dtiz8srr37VlzYPi0wZkLotkWEdxttAjAkiZUW6ksWP5jvXFKsG/uXsmdJ I3+fXc5Cbjo2z8rQHJgT14TpelTYndLV4qz35d2+PBa+ABqv9pDUNb2e9yS36gkLjN9a Nga/pRW1Yr2hBYA+QYBoTfVEgrjS70MW6BiRfpaEreJ8Lte8eki2lJ5QCHaGuP06boo+ oZmQ== X-Gm-Message-State: AElRT7GJSGZ1pmAM4F1vAjNGVcWEzFyHT0d6HG32n6INQyEZfQ5sRv5R Ct6m3Lu1ZvkUteBdUUbjPLV80ZVCr2iXh4DY4JuwIg== X-Google-Smtp-Source: AG47ELuY3zMhDYLdxn8kM8PHZuORyyUO1UuypNcHUanRbvUaWqG7I3A67XtOqXM9qxpytXe49gLSxsBVIx5QZkvL0nw= X-Received: by 10.46.41.156 with SMTP id p28mr21933648ljp.32.1522132551356; Mon, 26 Mar 2018 23:35:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.67.144 with HTTP; Mon, 26 Mar 2018 23:35:50 -0700 (PDT) From: Sivaprasanna Date: Tue, 27 Mar 2018 12:05:50 +0530 Message-ID: Subject: Proposal: Specifying range for List and Get processors To: dev@nifi.apache.org Content-Type: multipart/alternative; boundary="001a114b5bb806545505685f1a12" --001a114b5bb806545505685f1a12 Content-Type: text/plain; charset="UTF-8" I have been thinking about this lately. Now that NiFi has a wider presence on the big data stack, I feel it is better to have an optional range properties. Lets take, ListHDFS, ListAzureBlob or ListS3 for example, over time, the files accumulated under a particular location will grow exponentially. Having property descriptors that mention the upper and lower bound for the files, in this case, a time range - say 1/31/2016 - 2/26/2016, would be a nice to have feature. Thoughts? --001a114b5bb806545505685f1a12--