drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Yelchiyeva <arina.yelchiy...@gmail.com>
Subject Re: [DISCUSS] Drill 1.11.0 release
Date Wed, 19 Jul 2017 09:15:30 GMT
Thanks, Boaz!

Unfortunately, I still see the failures connected to DRILL-5669.
I'll share the details.

Kind regards
Arina

On Wed, Jul 19, 2017 at 2:00 AM, Boaz Ben-Zvi <bben-zvi@mapr.com> wrote:

> Hi Arina,
>
>      The three Jira’s are ready (+1, squashed, rebased, etc.):
>
> • DRILL-5616: Add memory checks, plus minor metrics changes
>     (Commit ID: c7ad56b7e075e84761896c9ce660b18b09d9e49a )
> • DRILL-5665: Add the option planner.force_2phase_aggr to override small
> inputs
>     (Commit ID: 1917ec084bdef74962370fbe5b253172c634c86e )
> • DRILL-5669: Add a configurable option for minimum memory allocation to
> buffered ops
>     (Commit ID: 4d7a3fe3e7ae52db57b0fd1f0619c6d1d64308c8 )
>
>   Thanks,
>
>          Boaz
>
> On 7/18/17, 3:04 AM, "Arina Yelchiyeva" <arina.yelchiyeva@gmail.com>
> wrote:
>
>     Hi Parth,
>
>     no problem, I did not start preparing the release since there is one
>     blocker has left:
>
>     *DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days)*
>
>
>     Kind regards
>     Arina
>
>     On Mon, Jul 17, 2017 at 11:26 PM, Parth Chandra <parthc@apache.org>
> wrote:
>
>     > Hi Arina,
>     >
>     >   I just realized that I did not push the C++ client fix (
> DRILL-5659) to
>     > master. I have just done so.
>     >
>     >   Sorry if I messed things up for the release, but we should get
> this fix
>     > in.
>     >
>     > Parth
>     >
>     > On Fri, Jul 14, 2017 at 8:15 AM, Paul Rogers <progers@mapr.com>
> wrote:
>     >
>     > > I looked into one NPE on CSV. Turns out it is already fixed by
> previous
>     > > 1.11 PRs, so we’re good.
>     > >
>     > > There is another reported by a user on the user list, but I’ve not
> heard
>     > > back about getting sample data. Let’s not hold the release for
> that.
>     > >
>     > > - Paul
>     > >
>     > > > On Jul 14, 2017, at 2:38 AM, Arina Yelchiyeva <
>     > > arina.yelchiyeva@gmail.com> wrote:
>     > > >
>     > > > *Blockers*
>     > > > DRILL-5659 - Parth (done)
>     > > > DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days)
>     > > >
>     > > > *Possible blockers*
>     > > > NPE on CSV - Paul can you please provide an update?
>     > > >
>     > > > Kind regards
>     > > > Arina
>     > > >
>     > > > On Thu, Jul 13, 2017 at 2:46 PM, Arina Yelchiyeva <
>     > > > arina.yelchiyeva@gmail.com> wrote:
>     > > >
>     > > >> *Blockers*
>     > > >> DRILL-5659 - problem is identified, Parth is working on the fix.
>     > > >>
>     > > >> *Not blockers*
>     > > >> DRILL-5660 - Vitalii
>     > > >> DRILL-5468 - Jinfeng
>     > > >>
>     > > >> *Possible blockers*
>     > > >> DRILL-5669 - Boaz Ben-Zvi / Paul
>     > > >> NPE on CSV - Paul
>     > > >>
>     > > >> Kind regards
>     > > >> Arina
>     > > >>
>     > > >>
>     > > >> On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <progers@mapr.com>
>     > wrote:
>     > > >>
>     > > >>> I withdraw my suggestion to fix DRILL-5660. Was a big hassle
> for us
>     > to
>     > > >>> track down the incompatibility the issue internally, but
> production
>     > > users
>     > > >>> probably don’t attempt to use one build of Drill with metadata
> files
>     > > >>> created from another. We can worry about the issue later only
> if a
>     > user
>     > > >>> actually runs into it.
>     > > >>>
>     > > >>> As suggested earlier, let’s just document this obscure case
in
> a
>     > > release
>     > > >>> note.
>     > > >>>
>     > > >>> - Paul
>     > > >>>
>     > > >>>> On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <jni@apache.org>
> wrote:
>     > > >>>>
>     > > >>>> Upgrade from 1.10 to 1.11 is irreversible only when parquet
> metadata
>     > > >>> cache
>     > > >>>> file is involved.
>     > > >>>>
>     > > >>>> I might be wrong, but I thought that if someone is running
on
>     > 1.10.0,
>     > > >>> and
>     > > >>>> he wants to go back to 1.0.0, where the first version
of
> parquet
>     > > >>> meatadata
>     > > >>>> cache was introduced, can we guarantee that Drillbit would
> work
>     > > >>> correctly?
>     > > >>>> I'm trying to understand what DRILL-5660 asks for is a
new
>     > > requirement,
>     > > >>> or
>     > > >>>> an requirement being there starting from Drill 1.0.0?
>     > > >>>>
>     > > >>>> Regression failures are normally regarded as release blocker.
>     > > >>>>
>     > > >>>>
>     > > >>>>
>     > > >>>> On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <
> progers@mapr.com>
>     > > wrote:
>     > > >>>>
>     > > >>>>> If we don’t do DRILL-5660 for 1.11 then we don’t
need to do
> it at
>     > > all,
>     > > >>> so
>     > > >>>>> that is a time savings. We could handle this via a
release
> note
>     > item
>     > > >>> that
>     > > >>>>> simply states that an upgrade from 1.10 to 1.11 is
> irreversible: we
>     > > do
>     > > >>> not
>     > > >>>>> support the ability to fall back to the old version.
> Everyone OK
>     > with
>     > > >>> that?
>     > > >>>>>
>     > > >>>>> We are seeing some stress test regression failures
that, I’m
> told,
>     > > will
>     > > >>>>> show up as JIRA entries shortly.
>     > > >>>>>
>     > > >>>>> - Paul
>     > > >>>>>
>     > > >>>>>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <jni@apache.org>
> wrote:
>     > > >>>>>>
>     > > >>>>>> I put some analysis in DRILL-5468.
>     > > >>>>>>
>     > > >>>>>> Also, IMHO, DRILL-5660 is not a release blocker.
>     > > >>>>>>
>     > > >>>>>>
>     > > >>>>>>
>     > > >>>>>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra
<
>     > parthc@apache.org>
>     > > >>>>> wrote:
>     > > >>>>>>
>     > > >>>>>>> Re DRILL-5634:
>     > > >>>>>>> From the Apache commons web page [1] -
>     > > >>>>>>>
>     > > >>>>>>> Please note that Apache Commons Crypto doesn't
implement
> the
>     > > >>>>> cryptographic
>     > > >>>>>>> algorithm such as AES directly.* It wraps
to Openssl or
> JCE which
>     > > >>>>> implement
>     > > >>>>>>> the algorithms.*
>     > > >>>>>>>
>     > > >>>>>>>
>     > > >>>>>>> Since we are cleared for using Openssl and
JCE, we are OK
> to
>     > > include
>     > > >>> the
>     > > >>>>>>> AES code in the UDFs.
>     > > >>>>>>>
>     > > >>>>>>>
>     > > >>>>>>> Parth
>     > > >>>>>>>
>     > > >>>>>>> [1] https://commons.apache.org/proper/commons-crypto/
>     > > >>>>>>>
>     > > >>>>>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva
<
>     > > >>>>>>> arina.yelchiyeva@gmail.com> wrote:
>     > > >>>>>>>
>     > > >>>>>>>> During today hangouts we agreed on approximate
cut-off
> date - 14
>     > > >>> July,
>     > > >>>>>>> 2017
>     > > >>>>>>>> (date may be shifted due to the blockers).
>     > > >>>>>>>> I'll start preparing test RC on Thursday
to solve any
> release
>     > > >>>>> preparation
>     > > >>>>>>>> issues beforehand.
>     > > >>>>>>>>
>     > > >>>>>>>> *Blockers*
>     > > >>>>>>>> DRILL-5660 - Vitalii
>     > > >>>>>>>>
>     > > >>>>>>>> *Possible blockers*
>     > > >>>>>>>> DRILL-5468 - Jinfeng
>     > > >>>>>>>> NPE on CSV source - Paul
>     > > >>>>>>>> DRILL-5659 - Parth
>     > > >>>>>>>>
>     > > >>>>>>>> *Will be included if done before the cut-off
date*
>     > > >>>>>>>> DRILL-5616 / DRILL-5665 - Boaz
>     > > >>>>>>>> DRILL-5634 - Parth will take a look at
Apache commons
>     > > implementation
>     > > >>>>> and
>     > > >>>>>>>> update.
>     > > >>>>>>>>
>     > > >>>>>>>> Kind regards
>     > > >>>>>>>> Arina
>     > > >>>>>>>>
>     > > >>>>>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles
Givre <
>     > cgivre@gmail.com>
>     > > >>>>> wrote:
>     > > >>>>>>>>
>     > > >>>>>>>>> Hi Arina,
>     > > >>>>>>>>> I can’t make the hangout, but we
have a few options.
>     > Basically,
>     > > >>> since
>     > > >>>>>>>> all
>     > > >>>>>>>>> the hashing algorithms have no export
restrictions, we
> could:
>     > > >>>>>>>>> 1.  Remove the AES encrypt/decrypt
until I can figure
> out what
>     > we
>     > > >>> have
>     > > >>>>>>> to
>     > > >>>>>>>>> do (and do it)
>     > > >>>>>>>>> 2.  I can try to research it today
or tomorrow to figure
> out
>     > > >>> exactly
>     > > >>>>>>> what
>     > > >>>>>>>>> we have to do to include AES.
>     > > >>>>>>>>> 3.  Remove the entire package
>     > > >>>>>>>>>
>     > > >>>>>>>>> I’d like to see the hashing functions
at least included
> because
>     > > >>> we’ve
>     > > >>>>>>> had
>     > > >>>>>>>>> a lot of requests for that functionality.
  I also don’t
> want
>     > > this
>     > > >>> to
>     > > >>>>>>>> hold
>     > > >>>>>>>>> things up, so please let me know your
preference.
> Regardless,
>     > > I’m
>     > > >>>>>>> going
>     > > >>>>>>>> to
>     > > >>>>>>>>> start trying to figure out the restrictions
so we’ll
> know for
>     > the
>     > > >>>>>>> future.
>     > > >>>>>>>>>
>     > > >>>>>>>>> Best,
>     > > >>>>>>>>> — C
>     > > >>>>>>>>>
>     > > >>>>>>>>>
>     > > >>>>>>>>>
>     > > >>>>>>>>>> On Jul 11, 2017, at 05:58, Arina
Yelchiyeva <
>     > > >>>>>>>> arina.yelchiyeva@gmail.com>
>     > > >>>>>>>>> wrote:
>     > > >>>>>>>>>>
>     > > >>>>>>>>>> @Jinfeng,
>     > > >>>>>>>>>> Let's discuss today on Drill hangout
final cut-off date.
>     > > >>>>>>>>>>
>     > > >>>>>>>>>> @Charles
>     > > >>>>>>>>>> Since there were questions about
AES in DRILL-5634,
> should we
>     > > >>> exclude
>     > > >>>>>>>> it
>     > > >>>>>>>>>> from the candidates for 1.11.0
release?
>     > > >>>>>>>>>>
>     > > >>>>>>>>>> @Boaz
>     > > >>>>>>>>>> Do we have any ETA when DRILL-5616
and DRILL-5665 will
> pass
>     > code
>     > > >>>>>>>> review?
>     > > >>>>>>>>>>
>     > > >>>>>>>>>> Also I believe we should include
DRILL-5660 where we
> need to
>     > > bump
>     > > >>> up
>     > > >>>>>>>>>> parquet metadata version.
>     > > >>>>>>>>>> @Vitalii
>     > > >>>>>>>>>> Do you have any ETA for the fix?
>     > > >>>>>>>>>>
>     > > >>>>>>>>>> Kind regards
>     > > >>>>>>>>>> Arina
>     > > >>>>>>>>>>
>     > > >>>>>>>>>> On Tue, Jul 11, 2017 at 6:02 AM,
Boaz Ben-Zvi <
>     > > bben-zvi@mapr.com>
>     > > >>>>>>>> wrote:
>     > > >>>>>>>>>>
>     > > >>>>>>>>>>> Hi Arina,
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>  Two PR have just been submitted,
relating to the Hash
> Aggr
>     > > >>>>>>> Spill:
>     > > >>>>>>>>>>> DRILL-5616 and DRILL-5665
.
>     > > >>>>>>>>>>> Can these be considered for
1.11 as well ?
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>  Thanks,
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>            Boaz
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>> On 7/10/17, 1:49 PM, "Paul
Rogers" <progers@mapr.com>
> wrote:
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>> DRILL-5601 is no longer a
blocker for 1.11; we’ll
> defer that
>     > > >>>>>>> until
>     > > >>>>>>>>>>> after the release to allow
others more time to work on
> other
>     > > >>>>>>> features
>     > > >>>>>>>>>>> rather than doing code reviews.
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>> - Paul
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>> On Jul 10, 2017, at 1:43
PM, Jinfeng Ni <
> jni@apache.org>
>     > > wrote:
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>> Hi Arina,
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>> It has been a while since
we discussed 1.11.0
> release.  Any
>     > > >>> update
>     > > >>>>>>>>>>> on the
>     > > >>>>>>>>>>>> new release plan?  Do
we have a cut-off date ?
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>> In the past, I believe
Drill release is a
> time-bounded, not
>     > > >>> feature
>     > > >>>>>>>>>>>> bounded. If certain fixes/features
could not make it
> within
>     > a
>     > > >>>>>>> certain
>     > > >>>>>>>>>>>> time-frame, we had better
defer them into future
> release,
>     > > unless
>     > > >>>>>>>>>>> those
>     > > >>>>>>>>>>>> fixes/features are regarded
as blocking issues for a
> new
>     > > >>> release.
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>> On Tue, Jul 4, 2017 at
5:14 AM, Arina Yelchiyeva <
>     > > >>>>>>>>>>> arina.yelchiyeva@gmail.com
>     > > >>>>>>>>>>>>> wrote:
>     > > >>>>>>>>>>>>
>     > > >>>>>>>>>>>>> Latest update:
>     > > >>>>>>>>>>>>>
>     > > >>>>>>>>>>>>> We wait for the following
Jiras before final cut off.
>     > > >>>>>>>>>>>>> 1. DRILL-5601  - Paul
>     > > >>>>>>>>>>>>> 2. DRILL-5420 - Kunal
>     > > >>>>>>>>>>>>> 3. DRILL-5634 - Charles
>     > > >>>>>>>>>>>>>
>     > > >>>>>>>>>>>>> Kind regards
>     > > >>>>>>>>>>>>> Arina
>     > > >>>>>>>>>>>>>
>     > > >>>>>>>>>>>>> On Fri, Jun 30, 2017
at 9:29 PM, Kunal Khatua <
>     > > >>> kkhatua@mapr.com>
>     > > >>>>>>>>>>> wrote:
>     > > >>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>> Parth's already
+1'd the process with a minor
> update. So,
>     > > >>> we're
>     > > >>>>>>>>>>> good to
>     > > >>>>>>>>>>>>> go
>     > > >>>>>>>>>>>>>> on DRILL-5420.
I'll change the label on that to
>     > > >>> ready-to-commit.
>     > > >>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>> <http://www.mapr.com/>
>     > > >>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>> ________________________________
>     > > >>>>>>>>>>>>>> From: Parth Chandra
<parthc@apache.org>
>     > > >>>>>>>>>>>>>> Sent: Thursday,
June 29, 2017 9:44:39 AM
>     > > >>>>>>>>>>>>>> To: dev@drill.apache.org
>     > > >>>>>>>>>>>>>> Subject: Re: [DISCUSS]
Drill 1.11.0 release
>     > > >>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>> I'm Ok with the
pcap format plugin if Pauls
> comments are
>     > > also
>     > > >>>>>>>>>>> addressed.
>     > > >>>>>>>>>>>>> If
>     > > >>>>>>>>>>>>>> the changes are
done before your cutoff deadline,we
> can
>     > > merge
>     > > >>> it
>     > > >>>>>>>>>>> in.
>     > > >>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>> On Thu, Jun 29,
2017 at 12:26 AM, Arina Yelchiyeva <
>     > > >>>>>>>>>>>>>> arina.yelchiyeva@gmail.com>
wrote:
>     > > >>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> Hi all,
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> here is the
list of Jira we are waiting before
> making
>     > final
>     > > >>> cut
>     > > >>>>>>>>>>> off:
>     > > >>>>>>>>>>>>>>> 1. DRILL-5601
 - Paul
>     > > >>>>>>>>>>>>>>> 2. DRILL-3640
 - Kunal
>     > > >>>>>>>>>>>>>>> 3. DRILL-5420
- Kunal
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> Please let
me know if there are any others Jiras we
>     > should
>     > > >>> wait
>     > > >>>>>>>>>>> for.
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> @Charles,
>     > > >>>>>>>>>>>>>>> you have mentioned
that you want to include some
> UDFs in
>     > > >>> Drill
>     > > >>>>>>>>>>> 1.11, do
>     > > >>>>>>>>>>>>>> you
>     > > >>>>>>>>>>>>>>> have any ETA
for the PR?
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> @Paul / Parth
>     > > >>>>>>>>>>>>>>> I see there
is active CR for pcap format PR, do we
> want
>     > to
>     > > >>>>>>>>>>> include it
>     > > >>>>>>>>>>>>> in
>     > > >>>>>>>>>>>>>>> this release
as well?
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> Kind regards
>     > > >>>>>>>>>>>>>>> Arina
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>> On Thu, Jun
29, 2017 at 10:05 AM, Arina Yelchiyeva
> <
>     > > >>>>>>>>>>>>>>> arina.yelchiyeva@gmail.com>
wrote:
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>> Parth,
thanks a lot the instructions.
>     > > >>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>> On Wed,
Jun 28, 2017 at 8:24 PM, Parth Chandra <
>     > > >>>>>>>>>>> parthc@apache.org>
>     > > >>>>>>>>>>>>>>> wrote:
>     > > >>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>> FWIW,
I put release instructions culled from
> gists
>     > > written
>     > > >>> by
>     > > >>>>>>>>>>>>> previous
>     > > >>>>>>>>>>>>>>>>> release
managers into an updated gist here [1]
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>> HTH
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>> Parth
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>> [1]
https://github.com/
> parthchandra/drill/wiki/Drill-
>     > > >>>>>>>>>>>>> Release-Process
>     > > >>>>>>>>>>>>>>>>> <https://github.com/
> parthchandra/drill/wiki/Drill-
>     > > >>>>>>>>>>> Release-Process>
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>> On
Tue, Jun 27, 2017 at 5:24 PM, Paul Rogers <
>     > > >>>>>>> progers@mapr.com>
>     > > >>>>>>>>>>>>>> wrote:
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
Hi Arina,
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
I have two projects in progress: one is a
> candidate
>     > for
>     > > >>> 1.11,
>     > > >>>>>>>>>>> the
>     > > >>>>>>>>>>>>>>> other
>     > > >>>>>>>>>>>>>>>>> is
>     > > >>>>>>>>>>>>>>>>>>
not.
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
The “managed external sort” is the good
> candidate for
>     > > >>> 1.11:
>     > > >>>>>>>>>>> The PR
>     > > >>>>>>>>>>>>>> for
>     > > >>>>>>>>>>>>>>>>>>
DRILL-5601 is being reviewed. Getting that in
> will
>     > allow
>     > > >>> us
>     > > >>>>>>> to
>     > > >>>>>>>>>>>>>> enable
>     > > >>>>>>>>>>>>>>>>> the
>     > > >>>>>>>>>>>>>>>>>>
“new” external sort by default.
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
The other project is to address Drill’s memory
>     > > >>> fragmentation
>     > > >>>>>>>>>>> issue
>     > > >>>>>>>>>>>>>>> with
>     > > >>>>>>>>>>>>>>>>>>
vectors over 16 MB in size. Two PRs have been
> open
>     > for a
>     > > >>>>>>> while.
>     > > >>>>>>>>>>>>> They
>     > > >>>>>>>>>>>>>>> are
>     > > >>>>>>>>>>>>>>>>>>
the first of several. Each takes a while to
> review.
>     > So,
>     > > >>> the
>     > > >>>>>>>>>>>>> project
>     > > >>>>>>>>>>>>>>> does
>     > > >>>>>>>>>>>>>>>>>>
not fit 1.11. The work can go into the next
> release
>     > > >>> instead.
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
Thanks,
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
- Paul
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>>
>     > > >>>>>>>>>>>>>
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>>>
>     > > >>>>>>>>>
>     > > >>>>>>>>>
>     > > >>>>>>>>
>     > > >>>>>>>
>     > > >>>>>
>     > > >>>>>
>     > > >>>
>     > > >>>
>     > > >>
>     > >
>     > >
>     >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message