Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C73D4200B67 for ; Tue, 2 Aug 2016 01:04:26 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C5C69160AA7; Mon, 1 Aug 2016 23:04:26 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EA633160A6C for ; Tue, 2 Aug 2016 01:04:25 +0200 (CEST) Received: (qmail 45496 invoked by uid 500); 1 Aug 2016 23:04:24 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 45486 invoked by uid 99); 1 Aug 2016 23:04:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Aug 2016 23:04:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 405A9C0D9B for ; Mon, 1 Aug 2016 23:04:24 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 5MRq_5Rkchph for ; Mon, 1 Aug 2016 23:04:22 +0000 (UTC) Received: from mail-qk0-f174.google.com (mail-qk0-f174.google.com [209.85.220.174]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 61B235F4EA for ; Mon, 1 Aug 2016 23:04:21 +0000 (UTC) Received: by mail-qk0-f174.google.com with SMTP id v123so27157910qkh.3 for ; Mon, 01 Aug 2016 16:04:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=5xwQyyJeK65Uz1lw1mCh3X13mC8r5SyA4LPlOmkLs3g=; b=SE/YdcJQXrsoFlnnSlWzYSjglvKjdLtAjXlVOIcYPfOl9lnHxUlAjQ4npXq5GFC6Ep +3vVzOElf+J+jgYXu4YRmlo4GGHlExWEZlrtRcRcSxjrD8RxDnIRY10/KrGxD/lc9qo+ FP4i6UavSP4NXV62LAlGs6Fg3phOZt5f7Re5XbE/IHVmq0u0gXDjgXMu49WdwPhYS1bb feC1tg6lvGXywk2PwKEp1UM6Nb6NlrBnAZprE+AgHw4r/09R5nqA2mktARlOWgOzUVl4 dxkCzETzQD4Vecd+3Zcn8m9sX1vXAN5AeJzsL3Cnt5HaqP3SG7A+qoQLBLe75YqFbRyt UmIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=5xwQyyJeK65Uz1lw1mCh3X13mC8r5SyA4LPlOmkLs3g=; b=aIM2uLHRXcJO9vTJTiRwtKssty1f3g6WNhouhOZmoW/JoYkr0TNdztSyHa/4YDvQk0 6muoH6ZsmcKBIpITl/vxg+hnRcnQLkEUBmJYXybTVUmsGnrvXRKADsAGhVoADsRYr/A3 wkBi6WR06pQfFRq5DfzGMrKZ9bzJLuCcejEMR3GLBvK/rTapgqCrE/IJLNbxVoszRET1 A9AEasaVdNV202XbltuyLKz+T3maLMJCyLp2/U+38SxRYP1dpembg4hBU9np4J5y98JG jivD9+oNFR3T40f0N7NIvfpYLJv4fxH6v7HmRP3CgeJYYP8eUUm522RRxpUL0ksspU2o BFMw== X-Gm-Message-State: AEkoouu9lVugnrTnUGi8v72TVLSLw6ZqDAsj+hJ7kFbWNb7U/zD0ZHgJth5y1+cYK5+ibYe8cqhkvgOvwFzRxA== X-Received: by 10.55.169.203 with SMTP id s194mr14182373qke.39.1470092660375; Mon, 01 Aug 2016 16:04:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.67.136 with HTTP; Mon, 1 Aug 2016 16:04:19 -0700 (PDT) In-Reply-To: <121BA448-6E2F-4209-B36C-76ADE0DB7CE5@gmail.com> References: <121BA448-6E2F-4209-B36C-76ADE0DB7CE5@gmail.com> From: Mich Talebzadeh Date: Tue, 2 Aug 2016 00:04:19 +0100 Message-ID: Subject: Re: How can I force Hive to start compaction on a table immediately To: user Content-Type: multipart/alternative; boundary=94eb2c06cc5acf083f05390a9ebd archived-at: Mon, 01 Aug 2016 23:04:27 -0000 --94eb2c06cc5acf083f05390a9ebd Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks Alan. One crude solution would be to copy data from the ACID table to a simple table and present that table to Spark to see the data. This is basically Spark optimiser issue not the engine itself My Hive runs on Spark query engine and all works fine there. HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=3DAAEAAAAWh2gBxianrbJd6= zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 1 August 2016 at 23:47, Alan Gates wrote: > There=E2=80=99s no way to force immediate compaction. If there are compa= ction > workers in the metastore that aren=E2=80=99t busy they should pick that u= p > immediately. But there isn=E2=80=99t an ability to create a worker threa= d and > start compacting. > > Alan. > > > On Aug 1, 2016, at 14:50, Mich Talebzadeh > wrote: > > > > > > Rather than queuing it > > > > hive> alter table payees COMPACT 'major'; > > Compaction enqueued. > > OK > > > > Thanks > > > > Dr Mich Talebzadeh > > > > LinkedIn > https://www.linkedin.com/profile/view?id=3DAAEAAAAWh2gBxianrbJd6zP6AcPCCd= OABUrV8Pw > > > > http://talebzadehmich.wordpress.com > > > > Disclaimer: Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising fro= m > such loss, damage or destruction. > > > > --94eb2c06cc5acf083f05390a9ebd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Alan.

One crude solut= ion would be to copy data from the ACID table to a simple table and present= that table to Spark to see the data.

This is basi= cally Spark optimiser issue not the engine itself

= My Hive runs on Spark query engine and all works fine there.

=
HTH

=

Dr Mich Talebzadeh

=C2=A0

LinkedIn =C2=A0https://www.linkedin.com/profile/view?id=3DAAEA= AAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

=C2=A0

http:= //talebzadehmich.wordpress.com


Disclaimer:=C2=A0Use = it=C2=A0at your own risk. Any and all responsibilit= y for any loss, damage or destruction of data or any other property which may arise from relying on this email= 9;s=C2=A0technical=C2=A0content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from = such loss, damage or destruction.

=C2=A0

<= font color=3D"#000000" face=3D"Times New Roman" size=3D"3">

On 1 August 2016 at 23:47, Alan Gates <a= lanfgates@gmail.com> wrote:
There=E2=80=99s no way to force immediate compaction.=C2=A0 If there are c= ompaction workers in the metastore that aren=E2=80=99t busy they should pic= k that up immediately.=C2=A0 But there isn=E2=80=99t an ability to create a= worker thread and start compacting.

Alan.

> On Aug 1, 2016, at 14:50, Mich Talebzadeh <mich.talebzadeh@gmail.com> wrote:
>
>
> Rather than queuing it
>
> hive> alter table payees COMPACT 'major';
> Compaction enqueued.
> OK
>
> Thanks
>
> Dr Mich Talebzadeh
>
> LinkedIn=C2=A0 https://www.linkedin.com/profile/view?id=3DAAEAAAAWh2gBxianrbJd6zP6AcPCCd= OABUrV8Pw
>
> http://talebzadehmich.wordpress.com
>
> Disclaimer: Use it at your own risk. Any and all responsibility for an= y loss, damage or destruction of data or any other property which may arise= from relying on this email's technical content is explicitly disclaime= d. The author will in no case be liable for any monetary damages arising fr= om such loss, damage or destruction.
>


--94eb2c06cc5acf083f05390a9ebd--