Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 90AA6200B61 for ; Tue, 26 Jul 2016 06:47:55 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 8F3DD160A8F; Tue, 26 Jul 2016 04:47:55 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D5666160A7D for ; Tue, 26 Jul 2016 06:47:54 +0200 (CEST) Received: (qmail 10357 invoked by uid 500); 26 Jul 2016 04:47:52 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 10346 invoked by uid 99); 26 Jul 2016 04:47:52 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Jul 2016 04:47:52 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 11AFCC2777 for ; Tue, 26 Jul 2016 04:47:52 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.198 X-Spam-Level: * X-Spam-Status: No, score=1.198 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id yjUJLOSljAXo for ; Tue, 26 Jul 2016 04:47:51 +0000 (UTC) Received: from mail-io0-f179.google.com (mail-io0-f179.google.com [209.85.223.179]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id 3F0105F1F5 for ; Tue, 26 Jul 2016 04:47:51 +0000 (UTC) Received: by mail-io0-f179.google.com with SMTP id b62so194378559iod.3 for ; Mon, 25 Jul 2016 21:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=NdxlwnsGreT1f5IeNNz92nYXNsHRiCr3Sdqby8yDFqs=; b=g0TZ0+90qfkibe0BNM2Kw+6Ruzqy9QLG2lujrC5Vlqo3cWZEUQZ/JT6NEiwLprzgvF +bb8Yrph9t/tkTvNpjdbGULwxGUxEoePMsxSHrUBQTc1UwolFFgQTDp/lCDxBH6jVGTv AmGUYOU2YGwJc8Y4eMCSIH0msUsrprM6wcrc6kBiNobWb72pJBgyNgiDyEJBKNmOOZk8 5LyACSeVfOTKem334e27izrVR4/TVBXwZipuoXtKWoqog/XKSL9NU6V80XXvkKfZB7Zt ihB89os/i/jpfGDJJB29Z8ffeprnktDsa7OSqw6+5ucMUY+rDoqRz1PYLcD85LaM3Gsu D4WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=NdxlwnsGreT1f5IeNNz92nYXNsHRiCr3Sdqby8yDFqs=; b=A9SIPIeCkgorsM2+kGHTJ10PJ+kMdrE4d8EF0nwBwUN4v0ovQNkwjswNOy90arHX0U 3f9gVWaBd+aSGZBPVfnWyWH2vgBtL+sDWXrjqdsyidpKQTBvxlrcanPBABjdqWK0wkSG 6I4i3fZKe1SPBNoJCQK8ynIHfhERST3ORQKWntuJBVtPoGF+zDSOd6swA1/1XKX2TYOO scwmdkLzkpM6JO66kgmcm0Qv2p2rQUvVYzG+ZuDwcdJV1SHilrmex3bUdJjuB324LTVQ zokQ+lBnTCMB0QCWmlBCb+bwfpIq6XL0wF8xHEsj7arPocqSwCH5wAi35h5VQzDARZgA xQUA== X-Gm-Message-State: AEkoouuqgwT5JlLtJ9BiTqXqJjm31j/FubvOXcz3WniXjuDHIh3V2wqeleH6Sweg5bY0bOTqHzK4/jiN7AF0Zw== X-Received: by 10.157.16.65 with SMTP id o1mr12093368oto.148.1469508470565; Mon, 25 Jul 2016 21:47:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.172.22 with HTTP; Mon, 25 Jul 2016 21:47:11 -0700 (PDT) From: Divya Gehlot Date: Tue, 26 Jul 2016 12:47:11 +0800 Message-ID: Subject: [How To :]Copy Merge + S3 File System To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11409d8061f3660538829af3 archived-at: Tue, 26 Jul 2016 04:47:55 -0000 --001a11409d8061f3660538829af3 Content-Type: text/plain; charset=UTF-8 Hi, val srcPath = s3n://bucket_name_1 > val dstPath =s3n://bucket_name_2 > val config = new Configuration() > val fs = FileSystem.get(URI.create(srcPath), config) > FileUtil.copyMerge(fs, new Path(srcPath), fs, new Path(dstPath), false, > config, null) I am trying to use Copymerge .. Its not throwing error but its not merging the files from one bucket to another . Would really appreciate the help. Thanks, Divya --001a11409d8061f3660538829af3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

val srcPath =3D =C2=A0s3n://b= ucket_name_1
val dstPath =3Ds3n://bucket_name_2=C2=A0
val config =3D = new Configuration()
v= al fs =3D FileSystem.get(URI.create(srcPath), config)
FileUtil.copyMerge(fs, new Path(srcPath), = fs, new Path(dstPath), false, config, null)

I am trying to use Copymerge .. Its not throwing error but its not me= rging the files from one bucket to another .

Would= really appreciate the help.

Thanks,
Div= ya
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0=C2=A0
--001a11409d8061f3660538829af3--