subversion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Foad <julian.f...@wandisco.com>
Subject Re: svn commit: r952493 - in /subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Date Tue, 15 Jun 2010 14:39:40 GMT
On Tue, 2010-06-08, c wrote:
> Author: gstein
> Date: Tue Jun  8 00:47:22 2010
> New Revision: 952493
> 
> URL: http://svn.apache.org/viewvc?rev=952493&view=rev
> Log:
> The query that we used to fetch all children in BASE_NODE and WORKING_NODE
> used a UNION between two SELECT statements. The idea was to have SQLite
> remove all duplicates for us in a single query. Unfortunately, this caused
> SQLite to create an ephemeral (temporary) table and place the results of
> each query into that table. It created an index to remove dupliates. Then
> it returned the values in that ephemeral table. For large numbers of
> nodes, the construction of the table and its index becomes very costly.
> 
> This change rebuilds gather_children() in wc_db.c to do the duplicate
> removal manually using a hash table. It does some simple scanning straight
> into an array when it knows duplicates cannot exist (one of BASE or
> WORKING is empty).
> 
> The performance problem of svn_wc__db_read_children() was first observed
> in issue #3499. The actual performance improvement is untested so far, but
> I'm assuming pburba can pick up this change and try in his scenario.
> 
> * subversion/libsvn_wc/wc_db.c:
>   (count_children): new helper to count the number of children of a given
>     PARENT_RELPATH within a specific table.
>   (add_children_to_hash): new helper to scan children, placing their names
>     into a hash table as keys (and the mapped values).
>   (union_children): new helper to scan both BASE_NODE and WORKING_NODE and
>     manually create a union of the resulting names using a hash table.
>   (single_table_children): new helper to return the children from a single
>     table.

Hi Greg.  Please could you copy these descriptions of the four new
functions into doc strings in the source file.

Thanks.
- Julian


>   (gather_children): rebuilt in terms of the above helpers
[...]

> Modified: subversion/trunk/subversion/libsvn_wc/wc_db.c
[...]
>  static svn_error_t *
> +count_children(int *count,
> +               int stmt_idx,
> +               svn_sqlite__db_t *sdb,
> +               apr_int64_t wc_id,
> +               const char *parent_relpath)
> +{
> +  svn_sqlite__stmt_t *stmt;
> +
> +  SVN_ERR(svn_sqlite__get_statement(&stmt, sdb, stmt_idx));
> +  SVN_ERR(svn_sqlite__bindf(stmt, "is", wc_id, parent_relpath));
> +  SVN_ERR(svn_sqlite__step_row(stmt));
> +  *count = svn_sqlite__column_int(stmt, 0);
> +  return svn_error_return(svn_sqlite__reset(stmt));
> +}
> +
> +
> +/* Each name is allocated in RESULT_POOL and stored into CHILDREN as a key
> +   pointed to the same name.  */
> +static svn_error_t *
> +add_children_to_hash(apr_hash_t *children,
> +                     int stmt_idx,
> +                     svn_sqlite__db_t *sdb,
> +                     apr_int64_t wc_id,
> +                     const char *parent_relpath,
> +                     apr_pool_t *result_pool)
[...]
> +
> +static svn_error_t *
> +union_children(const apr_array_header_t **children,
> +               svn_sqlite__db_t *sdb,
> +               apr_int64_t wc_id,
> +               const char *parent_relpath,
> +               apr_pool_t *result_pool,
> +               apr_pool_t *scratch_pool)
[...]
> +
> +static svn_error_t *
> +single_table_children(const apr_array_header_t **children,
> +                      int stmt_idx,
> +                      int start_size,
> +                      svn_sqlite__db_t *sdb,
> +                      apr_int64_t wc_id,
> +                      const char *parent_relpath,
> +                      apr_pool_t *result_pool)
[...]
>  /* */
> +static svn_error_t *
> +gather_children(const apr_array_header_t **children,
> +                svn_boolean_t base_only,
> +                svn_wc__db_t *db,
> +                const char *local_abspath,
> +                apr_pool_t *result_pool,
> +                apr_pool_t *scratch_pool)



Mime
View raw message