Return-Path: X-Original-To: apmail-avro-user-archive@www.apache.org Delivered-To: apmail-avro-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2FBB9EE12 for ; Wed, 23 Jan 2013 21:10:16 +0000 (UTC) Received: (qmail 25678 invoked by uid 500); 23 Jan 2013 21:10:15 -0000 Delivered-To: apmail-avro-user-archive@avro.apache.org Received: (qmail 25630 invoked by uid 500); 23 Jan 2013 21:10:15 -0000 Mailing-List: contact user-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@avro.apache.org Delivered-To: mailing list user@avro.apache.org Received: (qmail 25576 invoked by uid 99); 23 Jan 2013 21:10:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jan 2013 21:10:13 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jojspieg@gmail.com designates 209.85.212.174 as permitted sender) Received: from [209.85.212.174] (HELO mail-wi0-f174.google.com) (209.85.212.174) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jan 2013 21:10:05 +0000 Received: by mail-wi0-f174.google.com with SMTP id hq4so1125955wib.7 for ; Wed, 23 Jan 2013 13:09:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=RXzw0aOG4ZE2Ptwe5PJW8jcpoEVBNk7H3o88UaXKDAs=; b=uJdqrkl/Nu6piI7hteQ0dZ+7W1rz4bjgqTB71Fchuu3dYQJo43CM6sRKgrbWq5S/Gh K2SwFEiRAQMo7gUmICbrEU2riBV17h483p8grhS4S0nx8NejP30+4JwHQdELgL7J8kiK Pa7mLv8gySmyxy/TXKrLuoRuJ36LtU+drzybjCuFYlNi66sBGeoCWStD43vJH65uq8q9 Uv0VzozOrtwehwwZjVm7uoKInbjVnPc303nX4j1dNOJENHIWqwRl5NiDSYq3X2z2AAqZ qujelGiYDu4rnm5TZH2VVYYvLM0U0t2BitHmFzskj3zMW+6wpfK9rkrzXE6Qu+GknRlp BvMg== MIME-Version: 1.0 X-Received: by 10.194.58.13 with SMTP id m13mr5114897wjq.18.1358975385578; Wed, 23 Jan 2013 13:09:45 -0800 (PST) Received: by 10.180.89.106 with HTTP; Wed, 23 Jan 2013 13:09:45 -0800 (PST) Date: Wed, 23 Jan 2013 13:09:45 -0800 Message-ID: Subject: Synchronization Markers From: Josh Spiegel To: user@avro.apache.org Content-Type: multipart/alternative; boundary=047d7bacbdbe1dae8804d3fb1d17 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bacbdbe1dae8804d3fb1d17 Content-Type: text/plain; charset=ISO-8859-1 As I understand it, Avro container files contain synchronization markers every so often to support splitting the file. See: https://cwiki.apache.org/AVRO/faq.html#FAQ-Whatisthepurposeofthesyncmarkerintheobjectfileformat%3F (1) Why isn't the synchronization marker the same for every container file? (i.e. what is the point of generating it randomly every time) (2) Is it possible, at least in theory, for naturally occurring data to contain bytes that match the sync marker? If so, would this break synchronization? Thanks, Josh --047d7bacbdbe1dae8804d3fb1d17 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
As I understand it, Avro containe= r files contain synchronization markers=A0 every so often to support splitt= ing the file.=A0 See:
https://= cwiki.apache.org/AVRO/faq.html#FAQ-Whatisthepurposeofthesyncmarkerintheobje= ctfileformat%3F

(1) Why isn't the synchronization marker the same for e= very container file?=A0 (i.e. what is the point of generating it randomly e= very time)

(2) Is it possible, at least in theory, = for naturally occurring data to contain bytes that match the sync marker? I= f so, would this break synchronization?

Thanks,
Josh
--047d7bacbdbe1dae8804d3fb1d17--