hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuanbo Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12504) Ozone: Improve SQLCLI performance
Date Tue, 10 Oct 2017 13:45:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yuanbo Liu updated HDFS-12504:
    Attachment: HDFS-12504-HDFS-7240.001.patch

upload v1 patch for this JIRA. I've refactored the code in SQLCLi.java to improve the performance
of inserting sqlite db.

> Ozone: Improve SQLCLI performance
> ---------------------------------
>                 Key: HDFS-12504
>                 URL: https://issues.apache.org/jira/browse/HDFS-12504
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Yuanbo Liu
>              Labels: performance
>         Attachments: HDFS-12504-HDFS-7240.001.patch
> In my test, my {{ksm.db}} has *3017660* entries with total size of *128mb*, SQLCLI tool
runs over *2 hours* but still not finish exporting the DB. This is because it iterates each
entry and inserts that to another sqllite DB file, which is not efficient. We need to improve
this to be running more efficiently on large DB files.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message