hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuanbo Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12504) Ozone: Improve SQLCLI performance
Date Tue, 10 Oct 2017 13:45:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yuanbo Liu updated HDFS-12504:
------------------------------
    Attachment: HDFS-12504-HDFS-7240.001.patch

upload v1 patch for this JIRA. I've refactored the code in SQLCLi.java to improve the performance
of inserting sqlite db.

> Ozone: Improve SQLCLI performance
> ---------------------------------
>
>                 Key: HDFS-12504
>                 URL: https://issues.apache.org/jira/browse/HDFS-12504
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Yuanbo Liu
>              Labels: performance
>         Attachments: HDFS-12504-HDFS-7240.001.patch
>
>
> In my test, my {{ksm.db}} has *3017660* entries with total size of *128mb*, SQLCLI tool
runs over *2 hours* but still not finish exporting the DB. This is because it iterates each
entry and inserts that to another sqllite DB file, which is not efficient. We need to improve
this to be running more efficiently on large DB files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message