Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 73172200B4B for ; Thu, 21 Jul 2016 23:01:25 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 71A0D160A73; Thu, 21 Jul 2016 21:01:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B8E4B160A72 for ; Thu, 21 Jul 2016 23:01:24 +0200 (CEST) Received: (qmail 3092 invoked by uid 500); 21 Jul 2016 21:01:23 -0000 Mailing-List: contact dev-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list dev@drill.apache.org Delivered-To: moderator for dev@drill.apache.org Received: (qmail 98414 invoked by uid 99); 21 Jul 2016 21:00:00 -0000 X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.289 X-Spam-Level: X-Spam-Status: No, score=-4.289 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, RP_MATCHES_RCVD=-1.287, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=symc.onmicrosoft.com X-AuditID: d80ac3f3-f79126d0000018fe-0a-579137c57cbb DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=symc.onmicrosoft.com; s=selector1-symantec-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=uP7Hpnma2jhR0qQHNLBfDAxAfRJlF2AA5N6BWJs41J0=; b=ABZl58aywseJ0iYe6FmyiMf0yVKCRLUZS/FcUUik5FQr7GITo9o4dLZT42kUZP95FZTf3fshQJ48jzW9hcQRlpJMUmZavSaOUwPaCTZBU30m9DjHeBFb88CTzndH+OGBgfhtmF+QHqsWxkhUJ3INxy46VmjsgGIiCOdOfI9kzE8= From: Suma Cherukuri To: "user@drill.apache.org" , "dev@drill.apache.org" CC: Raghavendra Nandagopal Subject: Question about Apache Drill Use Case Thread-Topic: Question about Apache Drill Use Case Thread-Index: AQHR45LH58xIddOSVU6mvKU+Ty7Ucw== Date: Thu, 21 Jul 2016 20:59:33 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.6.2.160219 authentication-results: spf=none (sender IP is ) smtp.mailfrom=Suma_Cherukuri@symantec.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [155.64.38.20] x-ms-office365-filtering-correlation-id: 353b2919-a950-4fbd-c3c4-08d3b1a9eaa0 x-microsoft-exchange-diagnostics: 1;BY2PR16MB0757;6:EehLAD4ohQOYhKI//6Ev4i1vqdNGqyqVyGxUrlV1KFadlslfdHHgzRtBBpZ/rXMkIElQQxQ5Kv53xnP6bQIHqDNcFm/JAqrPdRZDEEU6NDRNwG8dkBEsp0Khr26loARzTwjHyAC1hKf064xYqU1qUHJsOhoNhH7TJU8HXy+sqlTTjhxEYY80S6x2kwZ0UABDftRin0vCZEKOxtc/4w2mDkJl/l5ygmKtERaoZvUPPWqEGBxPCKBrfaNqYWm5BIj7ElFqKL7qQ+E3Y6wACgm8Zv+n5QxbT4mi6+4BgJL/Yzp7NXEROXV4PiMB/xJqUTkq;5:rDKdBN2U8CxsXVFdayrNP1LiB2wsAW8mY+wXHXzENSyloXA8Ck8RasbXEyt75yJ7eqBEIAZL//vRdtLyPOUyliMcOPkG4wGSmLbytuAAfco7eLiMpU0ypm1L/8b28lTpy9cFNBv9f/2dx5ZIlmj6oQ==;24:+c28dFBOjsvUsnm8RWGtByQMmA406998CJ/0Kpam7p8Mqx4FECKWVz26NtQ0+Uv7pfgkxZ8NzVRHCNDWJGXohMnYyqoZHmFBufcL8wDNujI=;7:tHm1jTTMcfOcmjbuEd6Pfco4h8Wuf2W4lrofXhjCMDwq/8J+surOlVoO3F4uNC+4xiSGIjb+mwetS7FHmO6X8zKDLCq05gbyRidwiy5wTwsRs/PA5vmvjfdRZkfcJlwZneyls0MOZjZiPTOcf15dmx0br/k/+KH52TyOi086Rgrgz/ggsBy10noIG/iVEKP5OapkD4sqc0YGbF1s7kyklGGeVDHi5CD582Iv0o3BAgxxIq+VCbfIpadh4Qtit+Bn x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR16MB0757; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040130)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6041072)(6043046);SRVR:BY2PR16MB0757;BCL:0;PCL:0;RULEID:;SRVR:BY2PR16MB0757; x-forefront-prvs: 0010D93EFE x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(7916002)(189002)(199003)(84964002)(8936002)(77096005)(106116001)(83506001)(106356001)(122556002)(6116002)(586003)(5002640100001)(36756003)(189998001)(3846002)(3280700002)(2501003)(66066001)(8676002)(4001430100002)(102836003)(11100500001)(81166006)(68736007)(2900100001)(2906002)(81156014)(3660700001)(4326007)(7846002)(7736002)(99286002)(10290500002)(92566002)(86362001)(229853001)(101416001)(10400500002)(105586002)(16236675004)(87936001)(450100001)(54356999)(50986999)(107886002)(5001770100001)(4001350100001)(80792005)(97736004);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR16MB0757;H:CO1PR16MB1031.namprd16.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; received-spf: None (protection.outlook.com: symantec.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_D3B685C4CD12sumacherukurisymanteccom_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jul 2016 20:59:33.0213 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3b217a9b-6c58-428b-b022-5ad741ce2016 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR16MB0757 X-OriginatorOrg: symantec.com X-Brightmail-Tracker: H4sIAAAAAAAAA02Se0hTcRTH+213d9fp4DpnHq1ABv2Rzremf4SYf4j/TBKSUUR205uKj8k2 xQXh1MAHCGorcZQPyNBKLPGJk3S+avlqpROVopxo81Hmo0jQ7nYn+d/nnPM95/y+Pw7BFfXw fIiMHDWtzKGyJLgAG2yx3AkYjayWB7/t4kVNfP2BRw1aZvkxnPgS6wDvCrouuJRKZ2Xk08qg 6FuC9Bq9jZ/7wbNg27zI1aIj9wrkQgAZDqa2L1yWT8PM53a8AgkIEbmJoF5fwmeDDQR1s/cR G4wh2Pqz46yMIKjsH3YGawimTeOOACPLuaCz6ZwVHQea9mz/e4paX2H2nTgZCtrKJcd+MZkK b94fITtzyWiYNL9wsAcZAN17epzVhMLPsjpGTzAcCCv7HnbEyPPwrc2hFpKR8KRs3sGIcfTb 9JLDTvSCBWsDh3VKwlPDtNO1J3xfPuTZ2ZOZ2DvFMiJvwoGtBGM1Upi0WBHLvvBg2MRnWQY9 llmc5WI+TK34sxwHXcYqZz4TFv4uONkPykb0HPs3AGlCMLdRi1Uhqf7E+1i+AVsjn3C9w487 vKuzYmxeCo39v3CW/eFZ0zr3mCcGl5leguF4WDMoT0oaEfEcnVXnqUJU2WpFnjqXLggOC1Rp sinmmFICUxTZHYg5p3nB8E4vOjBcNiKSQBI34XhYtVzEo/IZpREBwZWIhS0RTEqYSmnu0kpF sjIvi1YZ0RkCk3gJ02RquYhMo9R0Jk3n0srjKodw8dGiNl7hqU3vfJ1sppzQXpQtiQ9FhkfC xseFqMH76sx6d2e5IUG13krV8DhD5sWEc3zF648c16Btzf7SxOquoNiAtcckN8f5asJ1Q4Vu fVBUkKRzmaNc98wdtaMRrolt4tvmxNIkfnOsNLZz7KGcuNAXsDFwLaG+fvVe6a4oTYKp0qkQ P65SRf0DUpRgLkoDAAA= archived-at: Thu, 21 Jul 2016 21:01:25 -0000 --_000_D3B685C4CD12sumacherukurisymanteccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Good Afternoon! I work as an engineer at Symantec. My team works on Multi-tenant Event Proc= essing System. Just a high level background, our customers write data to ka= fka brokers though agents like logstash and we process the events and save = the log data in Elastic Search and S3. Use Case: We have a use case where in we write batches of events to S3 when= file size limitation of 1MB (specific to our case) or a certain time thres= hold is reached. We are planning on merging the number of files specific t= o a folder into one single file based on either time limit such as every 24= hrs. We were considering various options available today and would like to know= if Apache Drill can be used to serve the purpose. Looking forward to hearing from you. Thank you Suma Cherukuri --_000_D3B685C4CD12sumacherukurisymanteccom_--