Return-Path: Delivered-To: apmail-jackrabbit-users-archive@minotaur.apache.org Received: (qmail 88033 invoked from network); 15 Jul 2009 08:57:47 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 15 Jul 2009 08:57:47 -0000 Received: (qmail 94250 invoked by uid 500); 15 Jul 2009 08:57:53 -0000 Delivered-To: apmail-jackrabbit-users-archive@jackrabbit.apache.org Received: (qmail 94227 invoked by uid 500); 15 Jul 2009 08:57:53 -0000 Mailing-List: contact users-help@jackrabbit.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@jackrabbit.apache.org Delivered-To: mailing list users@jackrabbit.apache.org Received: (qmail 94178 invoked by uid 99); 15 Jul 2009 08:57:53 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jul 2009 08:57:53 +0000 X-ASF-Spam-Status: No, hits=3.4 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [77.238.184.51] (HELO smtp120.mail.ukl.yahoo.com) (77.238.184.51) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 15 Jul 2009 08:57:43 +0000 Received: (qmail 27794 invoked from network); 15 Jul 2009 08:57:21 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.co.uk; h=Received:X-Yahoo-SMTP:X-YMail-OSG:X-Yahoo-Newman-Property:From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:X-Mailer:Thread-Index:Content-Language:x-cr-hashedpuzzle:x-cr-puzzleid; b=XGZoZ8eP4KYGQ5+D5IuwUYiaAZveTgrexu2Bg3Z/Pg89PsLuAoo8bWk31msYXuW395sj8TI9TcUsV231sTKfi1x1ZBJ/cROlml5NlMsM0IDftUE9fMUv1HmDXFdXhLke/behvxUFRJle7yn2fRi3fQu5M4BvfQWac+CBXJc+nuM= ; Received: from unknown (HELO MIQSHAUN) (sbarriba@87.84.49.84 with login) by smtp120.mail.ukl.yahoo.com with SMTP; 15 Jul 2009 08:57:21 -0000 X-Yahoo-SMTP: GBpOkfaswBDNrzFlsTrha4U9WPxCafTYZCtKv5cv6cVs X-YMail-OSG: 9ZHg9IkVM1nM3DGdHRaNcygMFs70ZK83GoSoWNXlwvBwkA3FtrE9Ft65LmOxKou2uLCavCFUdD8paMG61j436R2hbjx1myHzkmuPqr3D6Qvb7Sb7vO7fkP.kyeqrqhCPqAiBht9eTk47WXTWFzhf_iikarvHtGo0dgynSDwlTAYvaBjVIMMURMAI4WZjAEsuGJ1DtaOvVXnS.QFUw1kHvfuOHOkt5jXi3M31UX7yd_O9o5b6MHkpUgn3vNCZrSo3mLvnNPkULA3cZcL2Zv9NVw2fT0.Ozjvq0Cdk6._z3_CoMH_iK6TX2T1Z9tsXi0afFWkgR5_ncTGqp5w0afM- X-Yahoo-Newman-Property: ymail-3 From: "Shaun Barriball" To: Subject: Migrating a 15GB blob store to DataStore backed deployment Date: Wed, 15 Jul 2009 09:57:20 +0100 Message-ID: <008f01ca052a$438f4ab0$caade010$@co.uk> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0090_01CA0532.A553B2B0" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcoFKkHvRFlb2lqTT/+ahePf7XzqYA== Content-Language: en-gb x-cr-hashedpuzzle: Bek6 BjOV BmYf Cdtr D/uG EGGQ E3Ss E8Zl FL0X GwFS Hp6e HwjC IEBg IX3X JP5d KP6P;1;dQBzAGUAcgBzAEAAagBhAGMAawByAGEAYgBiAGkAdAAuAGEAcABhAGMAaABlAC4AbwByAGcA;Sosha1_v1;7;{47077FFE-6E7B-4D51-B941-0DBCF015EE01};cwBiAGEAcgByAGkAYgBhAEAAeQBhAGgAbwBvAC4AYwBvAC4AdQBrAA==;Wed, 15 Jul 2009 08:57:18 GMT;TQBpAGcAcgBhAHQAaQBuAGcAIABhACAAMQA1AEcAQgAgAGIAbABvAGIAIABzAHQAbwByAGUAIAB0AG8AIABEAGEAdABhAFMAdABvAHIAZQAgAGIAYQBjAGsAZQBkACAAZABlAHAAbABvAHkAbQBlAG4AdAA= x-cr-puzzleid: {47077FFE-6E7B-4D51-B941-0DBCF015EE01} X-Virus-Checked: Checked by ClamAV on apache.org ------=_NextPart_000_0090_01CA0532.A553B2B0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi all, We have a 15GB legacy JackRabbit repository currently putting Blobs in the database (MySql). This is causing the usual pain around a) bloated database backups, b) huge amounts of temporary files created due to Blob caching, c) lots of db IO and d) lots of lock contention (due to the IO). We therefore want to migrate this to a DataStore backed installation. http://wiki.apache.org/jackrabbit/DataStore mentions that: "Migration: currently there is no special mechanism to migrate data from a blob store to a data store. The only known way to convert is to export the data, and re-import into a new repository." .however a 15GB xml file is infeasible to import using "importxml" due to the memory requirements e.g. it seems to parse the XML in memory. I'd appreciate any bright ideas on how to achieve the migration? Thanks, Shaun ------=_NextPart_000_0090_01CA0532.A553B2B0--