Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 04BE211424 for ; Thu, 26 Jun 2014 04:19:18 +0000 (UTC) Received: (qmail 57537 invoked by uid 500); 26 Jun 2014 04:19:17 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 57483 invoked by uid 500); 26 Jun 2014 04:19:17 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 57471 invoked by uid 99); 26 Jun 2014 04:19:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Jun 2014 04:19:17 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sharninder@gmail.com designates 74.125.82.50 as permitted sender) Received: from [74.125.82.50] (HELO mail-wg0-f50.google.com) (74.125.82.50) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Jun 2014 04:19:12 +0000 Received: by mail-wg0-f50.google.com with SMTP id m15so2891432wgh.21 for ; Wed, 25 Jun 2014 21:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=xdLgm5ee0BfEmq3V6C17jhG3GY98Ds3KfHTYOPr13FA=; b=wfu2g4FdwXLpJyKh56uEhxJbpazMFsrbYQTwG7K4/GyO8N2J0COGJyh9LWO2f5IWN4 7EbFIe4jdmIDMgLPbNaKkL5EJRNDZIRzQC9NbxiREwDf9DbcJcQfHh9Mgp3jQL7f1Xwv GSfRQee5SSdVUXXlIwJgQdV78XNuG/dFfaGFW1yfBiC0EMFhnp2RPHUrWXp2zY8CFvSK z2ks+wGgByDHt0y3r4VNLxLDX7pYCZ62dnBYwJB++hABwqSIwNzyty3pQEXo3ExNdZ86 KeyeG1DlXX27+zh0Hi0l1MV8QMsAycDVuqNGsW6BlAa+SsETW6DdzGGIpj4jISnLWxIs RYyQ== MIME-Version: 1.0 X-Received: by 10.194.2.244 with SMTP id 20mr14109969wjx.26.1403756330982; Wed, 25 Jun 2014 21:18:50 -0700 (PDT) Received: by 10.194.43.9 with HTTP; Wed, 25 Jun 2014 21:18:50 -0700 (PDT) Date: Thu, 26 Jun 2014 09:48:50 +0530 Message-ID: Subject: End-to-End Encryption using flume From: Sharninder To: user@flume.apache.org Content-Type: multipart/alternative; boundary=047d7b343d8875a1d804fcb57d28 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b343d8875a1d804fcb57d28 Content-Type: text/plain; charset=ISO-8859-1 Hi, I'm trying to send a file (or events) over flume to a remote file store (could be HDFS too, later) with the only condition that the entire transmission be encrypted. The topology looks something like this: file source->encrypted avro sink -> avro source -> file sink This is failing at the second avro source and I have to add a decrypting source to enable this topology to work. Is this how flume encryption works? How can I make sure that the file/events encrypted at the first sink remains encrypted even when being written to disk at the end? Is this even possible ? Hope my description was clear enough? -- Sharninder --047d7b343d8875a1d804fcb57d28 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Hi,

I'= m trying to send a file (or events) over flume to a remote file store (coul= d be HDFS too, later) with the only condition that the entire transmission = be encrypted. The topology looks something like this:

file source->encrypted avro sink -> avro source -> file = sink

This is failing at the second avro source and I have to a= dd a decrypting source to enable this topology to work. Is this how flume e= ncryption works?

How can I make sure that the file/events encrypted at the first s= ink remains encrypted even when being written to disk at the end? Is this e= ven possible ?

Hope my description was clear enough?

Sharninder

--047d7b343d8875a1d804fcb57d28--