Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BF85C200D0C for ; Wed, 20 Sep 2017 08:28:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id BDC631609E1; Wed, 20 Sep 2017 06:28:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 10E4B1609E3 for ; Wed, 20 Sep 2017 08:28:05 +0200 (CEST) Received: (qmail 8960 invoked by uid 500); 20 Sep 2017 06:28:04 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 8949 invoked by uid 99); 20 Sep 2017 06:28:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Sep 2017 06:28:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 498B71A619B for ; Wed, 20 Sep 2017 06:28:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id h8nXmeiD8QLR for ; Wed, 20 Sep 2017 06:28:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 0070C60D3F for ; Wed, 20 Sep 2017 06:28:03 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 46E28E0E36 for ; Wed, 20 Sep 2017 06:28:02 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 643A4244FE for ; Wed, 20 Sep 2017 06:28:01 +0000 (UTC) Date: Wed, 20 Sep 2017 06:28:01 +0000 (UTC) From: "Weiwei Yang (JIRA)" To: hdfs-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HDFS-12504) Ozone: Improve SQLCLI performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 20 Sep 2017 06:28:06 -0000 Weiwei Yang created HDFS-12504: ---------------------------------- Summary: Ozone: Improve SQLCLI performance Key: HDFS-12504 URL: https://issues.apache.org/jira/browse/HDFS-12504 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Reporter: Weiwei Yang In my test, my {{ksm.db}} has *3017660* entries with total size of *128mb*, SQLCLI tool runs over *2 hours* but still not finish exporting the DB. This is because it iterates each entry and inserts that to another sqllite DB file, which is not efficient. We need to improve this to be running more efficiently on large DB files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org