Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Sat, 7 Apr 2012 01:01:29 +0000 (UTC)
From: "Vijay (Updated) (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <1266997258.848.1333760489587.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <2029376761.40.1325551220934.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (CASSANDRA-3690) Streaming CommitLog backup
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vijay updated CASSANDRA-3690:
-----------------------------

    Attachment: 0001-CASSANDRA-3690-v4.patch
    
> Streaming CommitLog backup
> --------------------------
>
>                 Key: CASSANDRA-3690
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3690
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>            Reporter: Vijay
>            Assignee: Vijay
>            Priority: Minor
>             Fix For: 1.1.1
>
>         Attachments: 0001-CASSANDRA-3690-v2.patch, 0001-CASSANDRA-3690-v4.patch, 0001-Make-commitlog-recycle-configurable.patch, 0002-support-commit-log-listener.patch, 0003-helper-jmx-methods.patch, 0004-external-commitlog-with-sockets.patch, 0005-cmmiting-comments-to-yaml.patch
>
>
> Problems with the current SST backups
> 1) The current backup doesn't allow us to restore point in time (within a SST)
> 2) Current SST implementation needs the backup to read from the filesystem and hence additional IO during the normal operational Disks
> 3) in 1.0 we have removed the flush interval and size when the flush will be triggered per CF, 
>           For some use cases where there is less writes it becomes increasingly difficult to time it right.
> 4) Use cases which needs BI which are external (Non cassandra), needs the data in regular intervals than waiting for longer or unpredictable intervals.
> Disadvantages of the new solution
> 1) Over head in processing the mutations during the recover phase.
> 2) More complicated solution than just copying the file to the archive.
> Additional advantages:
> Online and offline restore.
> Close to live incremental backup.
> Note: If the listener agent gets restarted, it is the agents responsibility to Stream the files missed or incomplete.
> There are 3 Options in the initial implementation:
> 1) Backup -> Once a socket is connected we will switch the commit log and send new updates via the socket.
> 2) Stream -> will take the absolute path of the file and will read the file and send the updates via the socket.
> 3) Restore -> this will get the serialized bytes and apply's the mutation.
> Side NOTE: (Not related to this patch as such) The agent which will take incremental backup is planned to be open sourced soon (Name: Priam).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira