From issues-return-18269-archive-asf-public=cust-asf.ponee.io@kylin.apache.org Mon Jun 25 05:29:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4FABA180657 for ; Mon, 25 Jun 2018 05:29:04 +0200 (CEST) Received: (qmail 39052 invoked by uid 500); 25 Jun 2018 03:29:03 -0000 Mailing-List: contact issues-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kylin.apache.org Delivered-To: mailing list issues@kylin.apache.org Received: (qmail 39043 invoked by uid 99); 25 Jun 2018 03:29:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Jun 2018 03:29:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id EE054181067 for ; Mon, 25 Jun 2018 03:29:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id dky5Gjy_obj2 for ; Mon, 25 Jun 2018 03:29:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 240D95F195 for ; Mon, 25 Jun 2018 03:29:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 5AEE4E062E for ; Mon, 25 Jun 2018 03:29:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 114A321829 for ; Mon, 25 Jun 2018 03:29:00 +0000 (UTC) Date: Mon, 25 Jun 2018 03:29:00 +0000 (UTC) From: "Shaofeng SHI (JIRA)" To: issues@kylin.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (KYLIN-3335) Add project & cube related info to the job id for better filtering MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KYLIN-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3335: -------------------------------- Fix Version/s: v2.5.0 > Add project & cube related info to the job id for better filtering > ------------------------------------------------------------------ > > Key: KYLIN-3335 > URL: https://issues.apache.org/jira/browse/KYLIN-3335 > Project: Kylin > Issue Type: Improvement > Reporter: Zhong Yanghong > Priority: Major > Fix For: v2.5.0 > > > Currently it's painful to search cube or project related jobs, since those infos are hidden in values. Especially, when users want to list job in a period under one project, by current design, all of the job output info have to be read into memory. If this kind of operation is done very often within a short period, it's easy to get OOM. > If the job id is prefixed with project and cube names, then we can push down prefix filters, which is efficient and safe. > This kind of change will cause backward compatibility issue. There're two ways to deal with this: > * Set a milestone with time tag, if a search relates to data earlier than this time, then just need to do one scan with prefix filter. Otherwise, two scans are needed. One with prefix filter and the other use current strategy. As time goes on, old job infos will be deleted. Once there's no data older than the time, only one scan is needed. > * Do migration for the old data once. -- This message was sent by Atlassian JIRA (v7.6.3#76005)