geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wes Williams (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GEODE-292) Optimize speed of backup/ restore cache snapshots
Date Fri, 28 Aug 2015 02:48:45 GMT

     [ https://issues.apache.org/jira/browse/GEODE-292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wes Williams updated GEODE-292:
-------------------------------
    Description: 
Backup/ restore using snapshots takes a very, very long time when:
1) There are a lot of PDXTypes, and
2) There are a lot of regions.
Specifically, it takes 35 minutes to restore only 50MB of unstructured JSON from snapshots.
In contrast, it takes only 3 minutes to reload all the data from scratch.

PROBLEM
CacheSnapshot loops all regions and saves all cache PDXTypes in every region.gfd. On restore,
it reloads all cache PDXTypes again for every region where they only need to be loaded once.
 

SOLUTION
This JIRA issue will create an option to save PDXTypes only once and reload them once and
store only data in the region snapshots. Existing functionality will remain for those who
want it.

  was:
Backup/ restore using snapshots takes a very, very long time when:
1) There are a lot of PDXTypes, and
2) There are a lot of regions.
Specifically, it takes 35 minutes to restore only 50MB of unstructured JSON from snapshots.
In contrast, it takes only 3 minutes to reload all the data from scratch.

PROBLEM
CacheSnapshot loops all regions and saves all PDXTypes in every region.gfd. On restore, it
reloads all PDXTypes for every region where they only need to be loaded once.  

SOLUTION
This JIRA issue will create an option to save PDXTypes only once and reload them once and
store only data in the region snapshots. Existing functionality will remain for those who
want it.


> Optimize speed of backup/ restore cache snapshots
> -------------------------------------------------
>
>                 Key: GEODE-292
>                 URL: https://issues.apache.org/jira/browse/GEODE-292
>             Project: Geode
>          Issue Type: Improvement
>          Components: persistence
>    Affects Versions: 1.0.0-incubating
>            Reporter: Wes Williams
>              Labels: backups, performance
>             Fix For: 1.0.0-incubating
>
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Backup/ restore using snapshots takes a very, very long time when:
> 1) There are a lot of PDXTypes, and
> 2) There are a lot of regions.
> Specifically, it takes 35 minutes to restore only 50MB of unstructured JSON from snapshots.
In contrast, it takes only 3 minutes to reload all the data from scratch.
> PROBLEM
> CacheSnapshot loops all regions and saves all cache PDXTypes in every region.gfd. On
restore, it reloads all cache PDXTypes again for every region where they only need to be loaded
once.  
> SOLUTION
> This JIRA issue will create an option to save PDXTypes only once and reload them once
and store only data in the region snapshots. Existing functionality will remain for those
who want it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message