www-infrastructure-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tony Stevenson (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (INFRA-4380) Import Flex JIRA
Date Tue, 05 Feb 2013 07:28:15 GMT

    [ https://issues.apache.org/jira/browse/INFRA-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13571113#comment-13571113
] 

Tony Stevenson edited comment on INFRA-4380 at 2/5/13 7:26 AM:
---------------------------------------------------------------

{quote}

Alex,

So I took your zip file onto a mac exploded it using the UI, and using unzip on the terminal.

They produced exactly the same output.  

I then did a clean install of JIRA 5.0.4, and started the project import.  See the attached
catalina log file. This is from a clean start, and attempted import. 

root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | wc -l
28267

That shows we have 28267 missing files.  Something isn't right here. 

root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | head -n3


2013-02-04 10:32:23,283 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143
/secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler]
The attachment 'rotateBug.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/10000'.
It will not be imported.

2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143
/secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler]
The attachment 'workaround.mxml' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305/10001'.
It will not be imported.

2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143
/secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler]
The attachment 'BusyCursorTest.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313/10002'.
It will not be imported.

That is 3 missing files at the start of the import. 

root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/
    
total 628K
drwxr-xr-x     2 root root 4.0K Feb  4 09:52 .
drwxr-xr-x 12686 root root 356K Feb  4 09:56 ..
-rwxr-xr-x     1 root root 261K Feb  4 09:52 10186

root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305
total 368K
drwxr-xr-x     2 root root 4.0K Feb  4 09:51 .
drwxr-xr-x 12686 root root 356K Feb  4 09:56 ..
-rwxr-xr-x     1 root root 1.8K Feb  4 09:51 10187

root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313
total 380K
drwxr-xr-x     2 root root 4.0K Feb  4 09:53 .
drwxr-xr-x 12686 root root 356K Feb  4 09:56 ..
-rwxr-xr-x     1 root root  14K Feb  4 09:53 10188


As you can see none of the blob objects in the zip file match. None. Did you do some kind
of transforms on this data?  
The file that the import is looking for is taken from the xml import file (in the zip).  Did
you import the data into another version of jira to export only the FLEX project, i.e. filter
out others.  If so you have to import the attachments into that instance as they will get
renumbered starting at 10000 - if not the attachments we have are from the original and mismatch
your exported zip/xml file. 

As it stands I cannot use the attachments. 

The attachments on disk do not use files names as per the source file.  They are stored as
blobs on disk starting at 10000, and the filename is stored in SQL. 


Alex, as far as I can tell.  You did the export from Adobe, into a staging instance and then
exported again after some filtering I suspect.  That will only work if you import the the
files as part of your import/export process. 

So far I have spent over 75 hours on this since the start of the year. Is there anyway we
can start the process again?  You get a clean dump and attachments? If so please can we get
it as a tar.gz ? 

I hope this all makes sense.  If not just shout. 

{quote}
                
      was (Author: pctony):
    Alex,

So I took your zip file onto a mac exploded it using the UI, and using unzip on the terminal.

They produced exactly the same output.  

I then did a clean install of JIRA 5.0.4, and started the project import.  See the attached
catalina log file. This is from a clean start, and attempted import. 

root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | wc -l
28267

That shows we have 28267 missing files.  Something isn't right here. 

root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | head -n3


2013-02-04 10:32:23,283 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143
/secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler]
The attachment 'rotateBug.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/10000'.
It will not be imported.

2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143
/secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler]
The attachment 'workaround.mxml' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305/10001'.
It will not be imported.

2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143
/secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler]
The attachment 'BusyCursorTest.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313/10002'.
It will not be imported.

That is 3 missing files at the start of the import. 

root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/
    
total 628K
drwxr-xr-x     2 root root 4.0K Feb  4 09:52 .
drwxr-xr-x 12686 root root 356K Feb  4 09:56 ..
-rwxr-xr-x     1 root root 261K Feb  4 09:52 10186

root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305
total 368K
drwxr-xr-x     2 root root 4.0K Feb  4 09:51 .
drwxr-xr-x 12686 root root 356K Feb  4 09:56 ..
-rwxr-xr-x     1 root root 1.8K Feb  4 09:51 10187

root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313
total 380K
drwxr-xr-x     2 root root 4.0K Feb  4 09:53 .
drwxr-xr-x 12686 root root 356K Feb  4 09:56 ..
-rwxr-xr-x     1 root root  14K Feb  4 09:53 10188


As you can see none of the blob objects in the zip file match. None. Did you do some kind
of transforms on this data?  
The file that the import is looking for is taken from the xml import file (in the zip).  Did
you import the data into another version of jira to export only the FLEX project, i.e. filter
out others.  If so you have to import the attachments into that instance as they will get
renumbered starting at 10000 - if not the attachments we have are from the original and mismatch
your exported zip/xml file. 

As it stands I cannot use the attachments. 

The attachments on disk do not use files names as per the source file.  They are stored as
blobs on disk starting at 10000, and the filename is stored in SQL. 


Alex, as far as I can tell.  You did the export from Adobe, into a staging instance and then
exported again after some filtering I suspect.  That will only work if you import the the
files as part of your import/export process. 

So far I have spent over 75 hours on this since the start of the year. Is there anyway we
can start the process again?  You get a clean dump and attachments? If so please can we get
it as a tar.gz ? 

I hope this all makes sense.  If not just shout. 
                  
> Import Flex JIRA
> ----------------
>
>                 Key: INFRA-4380
>                 URL: https://issues.apache.org/jira/browse/INFRA-4380
>             Project: Infrastructure
>          Issue Type: Task
>      Security Level: public(Regular issues) 
>          Components: JIRA
>            Reporter: Alex Harui
>            Assignee: Tony Stevenson
>         Attachments: catalina.out, flex-20120201.xml
>
>
> Files representing the Adobe Flex JIRA database has been made available to Tony Stevenson
and infrastructure@a.o via DropBox.
> The FLEX.zip is the 6GB zip of the attachments.  According to an MD5 app I got off the
internet, the checksum is: 12c4b40f0ada7aa7145e52d338399441
> The FlexSDKENTDMV.zip is the import file.  It's checksum is: a8118ac7e2feb452eb6281091ec09582
> Please perform a Project Import of these files into the FLEX project at issues.a.o.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message