db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rick Hillegas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DERBY-6136) Create a custom/optional tool for dumping the data in a corrupted database.
Date Fri, 05 Apr 2013 14:35:15 GMT

     [ https://issues.apache.org/jira/browse/DERBY-6136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rick Hillegas updated DERBY-6136:
---------------------------------

    Attachment: DataFileVTI.java

Attaching a new version of DataFileVTI. This version makes the following changes:

1) Skips records which are marked as deleted.

2) Turns errors into SQLWarnings so that problems will still be reported as reading continues.
The warnings percolate up to the user only if you are using a codeline improved by derby-6151-01-aa-passBackWarnings.diff.

3) Removed dead code and reorganized to improve readability.

I have run DataFileVTI successfully against corrupt heaps. That is, against heaps which spew
hex dumps when you try to select from them directly. Here is an ij example of using DataFileVTI
to read a corrupt heap:

call syscs_util.syscs_register_tool
(
    'customTool',
    true,
    'RawDBReader', 
    'CONTROL11',
    'RAW11_',
    '/Users/rh161140/derby/mainline/corruptEncryptedDB',
    'bootPassword=mysecretpassword',
    'APP',
    null
);
0 rows inserted/updated/deleted
ij(CONNECTION2)> select * from raw11_app.t1 where a > 37 and a < 50;
A          |B                                                                            
                                                  |C                                     
                                                                                         
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
38         |abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs&|abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs&
39         |abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs&|abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs&
48         |abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs&|abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs&
WARNING (no SQLState): Error reading field data. Offset = 366, length = 290, datatype = VARCHAR(1000):
Field 2 in record 0 on page 5 in file c490.dat: java.io.UTFDataFormatException: null
WARNING (no SQLState): Error reading field data. Offset = 671, length = 1823514738, datatype
= VARCHAR(1000): Field 1 in record 1 on page 5 in file c490.dat: java.io.IOException: Negative
seek offset
WARNING (no SQLState): Error reading field data. Offset = 685, length = 36, datatype = VARCHAR(1000):
Field 2 in record 1 on page 5 in file c490.dat: java.io.EOFException: null
WARNING (no SQLState): java.lang.ArrayIndexOutOfBoundsException: null
49         |abcdefghijklmnopqrstuvwxyz0123456789                                         
                                                  |abcdefghijklmnopqrstuvwxyz0123456789  
                                                                                         

4 rows selected


I think that these tools are ready for experimental use by others. Please tell me about the
problems you find, so that I can improve these tools. Thanks.

                
> Create a custom/optional tool for dumping the data in a corrupted database.
> ---------------------------------------------------------------------------
>
>                 Key: DERBY-6136
>                 URL: https://issues.apache.org/jira/browse/DERBY-6136
>             Project: Derby
>          Issue Type: Improvement
>          Components: Tools
>    Affects Versions: 10.11.0.0
>            Reporter: Rick Hillegas
>         Attachments: DataFileVTI.java, DataFileVTI.java, DataFileVTI.java, DataFileVTI.java,
dataFileVTI.sql, RawDBReader.java, RawDBReader.java
>
>
> It would be useful to have a tool for dumping the data in a corrupted database. This
could start out as a custom tool. After we debug the tool and get some experience with it,
we can consider promoting it to be a (possibly undocumented) optional tool which we ship with
the product. I think the tool should have the following behavior:
> 1) The tool should not subvert the security of the corrupted database. If the corrupted
database is password-protected, then you would need to present its DBO's credentials in order
to use the tool. Naturally, an encryption key would have to be presented in order to decode
an encrypted database.
> 2) The tool should not stop reading a table when it hits a corrupt record. Instead, the
tool should soldier on and collect a list of warnings on bad records.
> Such a tool would be useful in situations where some part of a heap table is corrupt
but the following heap conglomerates are intact:
> i) SYSSCHEMAS
> ii) SYSTABLES
> iii) SYSCONGLOMERATES
> iv) SYSCOLUMNS
> v) property conglomerate
> Such a tool would be useful for some situations where data can't be dumped even after
you delete the log files in order to short-circuit recovery.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message