hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Drob <md...@apache.org>
Subject Re: [VOTE] First release candidate for HBase 1.1.13 (RC0) is available
Date Mon, 11 Dec 2017 16:20:46 GMT
Yea, this candidate is fine to promote from my perspective and given the
other votes cast. Thanks for putting this together, Nick!

On Sun, Dec 10, 2017 at 7:11 PM, Nick Dimiduk <ndimiduk@apache.org> wrote:

> At close of the period, this VOTE has received 3x binding +1's, a
> non-binding +1, and a non-binding -1, with no other votes cast.
>
> My understanding is that the issues raised by the non-binding -1 are to be
> taken as guidance for subsequent release lines and do not impact the
> standing of this candidate.
>
> Mike, is that view consistent with your intentions?
>
> Thanks,
> Nick
>
>
> On Fri, Dec 8, 2017 at 9:00 PM, Nick Dimiduk <ndimiduk@apache.org> wrote:
>
> > +1
> >
> > - verified tarballs vs public key on people.apache.org.
> > - extracted bin tgz:
> >   - inspect structure. look good.
> >   - with jdk1.8.0_65:
> >     - run LoadTestTool against standalone bin tgz with FAST_DIFF block
> > encoder and ROWCOL blooms. No issues, logs look good.
> >     - poked around webUI. looks good.
> >   - load the site, browsed book.
> > - extracted src tgz:
> >   - inspect structure. look good.
> >   - run LoadTestTool against standalone built from src tgz with FAST_DIFF
> > block encoder and ROWCOL blooms. No issues, logs look good.
> >   - poked around webUI. looks good.
> > - ran the hbase-downstreamer project vs. the staged maven repository.
> > tests pass.
> >
> > On Thu, Dec 7, 2017 at 1:44 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> +1
> >>
> >> Checked sums and signatures: ok
> >> Ran unit tests: passed
> >> Started standalone cluster and did some basic operations
> >>
> >> On Thu, Dec 7, 2017 at 1:14 PM, Andrew Purtell <apurtell@apache.org>
> >> wrote:
> >>
> >> > +1
> >> >
> >> > Checked sums and signatures: ok
> >> > Checked compat report: ok
> >> > RAT check passed: ok (7u80)
> >> > Built from source: ok (7u80)
> >> > Unit tests pass: ok (8u131)
> >> > 1M row LTT: ok (8u131)
> >> >
> >> >
> >> > On Thu, Dec 7, 2017 at 8:40 AM, Nick Dimiduk <ndimiduk@apache.org>
> >> wrote:
> >> >
> >> > > No one has voted a binding -1 with actionable changes, so as far as
> >> I'm
> >> > > concerned this RC remains valid. If people need more time, we can
> >> extend
> >> > > this vote.
> >> > >
> >> > > Thanks,
> >> > > Nick
> >> > >
> >> > > On Thu, Dec 7, 2017 at 8:07 AM, Ted Yu <yuzhihong@gmail.com>
wrote:
> >> > >
> >> > > > Nick:
> >> > > > Originally you set tomorrow as deadline.
> >> > > >
> >> > > > Is there a new RC coming out (w.r.t. Mike's comment) ?
> >> > > >
> >> > > > Cheers
> >> > > >
> >> > > > On Mon, Dec 4, 2017 at 8:37 PM, Nick Dimiduk <ndimiduk@apache.org
> >
> >> > > wrote:
> >> > > >
> >> > > > > Mike:
> >> > > > >
> >> > > > > > Do you plan to make a human-readable set of release
notes in
> >> > addition
> >> > > > to
> >> > > > > the list of JIRA issues resolved?
> >> > > > >
> >> > > > > Not as such. For all branch-1.1 releases, I've written up
a
> little
> >> > > > > human-friendly summary in the ANNOUNCE email. Basically,
> >> expanding on
> >> > > the
> >> > > > > list of JIRA tickets I highlight in the RC notes to include
> their
> >> > full
> >> > > > > ticket summaries. I haven't followed the details of the
> branch-1.4
> >> > > > release
> >> > > > > line, so I'm not sure what additional information you might
be
> >> hoping
> >> > > > for.
> >> > > > >
> >> > > > > > tar missing hbase-native-client (present in tag)
> >> > > > >
> >> > > > > That's been the case since rel/1.1.0. We as a community
have
> >> never
> >> > > > shipped
> >> > > > > a binary native client in this release line and we've never
> >> claimed
> >> > > that
> >> > > > > the native sources packaged herein are ready for production
> >> > > consumption.
> >> > > > > They probably should have been dropped from the branch before
> >> initial
> >> > > > > release, but that was not done. I have no objection to dropping
> >> them
> >> > > > from a
> >> > > > > branch-1.1 release; from the git log, I see no commit activity
> to
> >> > that
> >> > > > > module since Jan 2014. I don't see any of this as a blocker
for
> >> this
> >> > > RC.
> >> > > > >
> >> > > > > > WARNING! HBase file layout needs to be upgraded ...
> >> > > > >
> >> > > > > When I test these RC's on a Mac, I explicitly set hbase.tmp.dir
> >> to a
> >> > > > > location specific to the candidate I've unpacked. This has
the
> >> > benefit
> >> > > > > avoiding cross-version conflicts and other weirdness of
Mac tmp
> >> > > directory
> >> > > > > management. For instance,
> >> > > > >
> >> > > > > <configuration>
> >> > > > >
> >> > > > > <property><name>hbase.tmp.dir</name><value>/tmp/hbase-1.1.
> >> > > > > 13/tmp</value></property>
> >> > > > > </configuration>
> >> > > > >
> >> > > > > Peter:
> >> > > > >
> >> > > > > > In the logs I saw this line. Source code repository
URL looks
> >> > > > incorrect.
> >> > > > > > 2017-12-04 10:13:27,028 INFO  [main] util.VersionInfo:
Source
> >> code
> >> > > > > repository *git://diocles.local/Volumes/hbase-1.1.13/hbase*
> >> > > > > revision=c64bf8a9f35352cd504f2b8f4b02f9148cf45ab6
> >> > > > >
> >> > > > > Looking through the log, HBASE-16538 /
> >> > > > > 851c89af6ef9a78e2e3bc9ad3153367e85731c81 looks suspicious.
It
> >> looks
> >> > > like
> >> > > > > that change first shipped in 1.1.7. Indeed, I see the equivelant
> >> line
> >> > > in
> >> > > > > the binary release of 1.1.12.
> >> > > > >
> >> > > > > On Mon, Dec 4, 2017 at 4:12 PM, Stack <stack@duboce.net>
wrote:
> >> > > > >
> >> > > > > > On Mon, Dec 4, 2017 at 11:52 AM, Andrew Purtell <
> >> > apurtell@apache.org
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > > I think this can be fixed by using `git archive`
to
> generate
> >> > the
> >> > > > src
> >> > > > > > tar
> >> > > > > > > instead of the src assembly.
> >> > > > > > >
> >> > > > > > > We could update the make_rc.sh script to create
the source
> >> > tarball
> >> > > in
> >> > > > > > this
> >> > > > > > > way. Would you be willing to make a patch for
that?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > (I think you fellows already figured this going by
JIRA
> >> > movement....)
> >> > > > > >
> >> > > > > > HBASE-19152 "Update refguide 'how to build an RC' and
the
> >> > make_rc.sh
> >> > > > > > script" changed make_rc.sh to use git archive. Wasn't
> backported
> >> > > > though.
> >> > > > > > S
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > > For release 1.4.0 and up though I propose we remove
> >> > > > hbase-native-client
> >> > > > > > as
> >> > > > > > > unmaintained. It can be resurrected as an independent
> project
> >> on
> >> > > > GitHub
> >> > > > > > if
> >> > > > > > > any interested parties show up.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Mon, Dec 4, 2017 at 11:45 AM, Mike Drob <
> mdrob@apache.org>
> >> > > wrote:
> >> > > > > > >
> >> > > > > > > > I think I used some shorthand earlier and
ambiguously
> >> > represented
> >> > > > my
> >> > > > > > > > problem. The issue is that native is missing
from the
> >> _source_
> >> > > > > > tarball. I
> >> > > > > > > > completely understand why it would be missing
from the
> >> binary
> >> > > > > tarball,
> >> > > > > > > > maintaining native binaries is a huge hassle
and not in
> our
> >> > > scope,
> >> > > > > IMO.
> >> > > > > > > >
> >> > > > > > > > However, I think it is fair for users to
expect that they
> >> can
> >> > > > > download
> >> > > > > > > our
> >> > > > > > > > source release, and build a native client
from there. Or
> at
> >> > least
> >> > > > it
> >> > > > > > will
> >> > > > > > > > have native code. They shouldn't need to
go to git to the
> >> the
> >> > > > native
> >> > > > > > > source
> >> > > > > > > > code.
> >> > > > > > > >
> >> > > > > > > > I think this can be fixed by using `git archive`
to
> generate
> >> > the
> >> > > > src
> >> > > > > > tar
> >> > > > > > > > instead of the src assembly.  The other option
is to
> remove
> >> the
> >> > > > > > > > hbase-native-client directory from branch-1.1
and then
> >> there's
> >> > no
> >> > > > > > > > divergence between src tar and src tag.
> >> > > > > > > >
> >> > > > > > > > Either is fine with me.
> >> > > > > > > >
> >> > > > > > > > On Mon, Dec 4, 2017 at 1:36 PM, Andrew Purtell
<
> >> > > > apurtell@apache.org>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > I propose to eject hbase-native-client
to GitHub on
> >> > HBASE-19419
> >> > > > > > > > >
> >> > > > > > > > > On Mon, Dec 4, 2017 at 11:27 AM, Andrew
Purtell <
> >> > > > > apurtell@apache.org
> >> > > > > > >
> >> > > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > hbase-native-client isn't hooked
up to the build. We
> >> don't
> >> > > > have a
> >> > > > > > > > > 'native'
> >> > > > > > > > > > profile like Hadoop that recurses
into native
> component
> >> > > > > directories
> >> > > > > > > and
> >> > > > > > > > > > invokes cmake. This won't be available
in any RC any
> RM
> >> > would
> >> > > > > > > generate.
> >> > > > > > > > > > Does one even make sense? Binaries
built against what?
> >> What
> >> > > do
> >> > > > > you
> >> > > > > > > > > propose?
> >> > > > > > > > > >
> >> > > > > > > > > > On Mon, Dec 4, 2017 at 9:14 AM,
Mike Drob <
> >> > mdrob@apache.org>
> >> > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > >> Thanks for the pointer, Andrew.
This worked for me
> and
> >> I
> >> > was
> >> > > > > able
> >> > > > > > to
> >> > > > > > > > > start
> >> > > > > > > > > >> hbase now and can resume testing.
> >> > > > > > > > > >>
> >> > > > > > > > > >> I didn't mean to imply that
Nick needs to create
> human
> >> > > release
> >> > > > > > > notes,
> >> > > > > > > > > but
> >> > > > > > > > > >> wanted to know if he had considered
it in light of
> the
> >> > plan
> >> > > to
> >> > > > > do
> >> > > > > > so
> >> > > > > > > > for
> >> > > > > > > > > >> 1.4.0. I think it's a nice
to have, but definitely
> not
> >> > > > critical.
> >> > > > > > > > > >>
> >> > > > > > > > > >> (I'm still -1 due to the tar/tag
discrepancy,
> however.)
> >> > > > > > > > > >>
> >> > > > > > > > > >> On Mon, Dec 4, 2017 at 11:05
AM, Andrew Purtell <
> >> > > > > > > > > andrew.purtell@gmail.com
> >> > > > > > > > > >> >
> >> > > > > > > > > >> wrote:
> >> > > > > > > > > >>
> >> > > > > > > > > >> > I think you need to delete
$TMPDIR/hbase-mdrob (or
> >> > > whatever
> >> > > > is
> >> > > > > > > your
> >> > > > > > > > > >> > username) before testing
the binaries. Looks like
> >> you've
> >> > > > been
> >> > > > > > > > testing
> >> > > > > > > > > a
> >> > > > > > > > > >> > later version and it's
left data behind.
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > We have not produced release
notes beyond the JIRA
> >> > > generated
> >> > > > > > ones
> >> > > > > > > > for
> >> > > > > > > > > >> > prior releases. We will
soon. This is something I
> >> agreed
> >> > > to
> >> > > > > for
> >> > > > > > > > > 1.4.0. I
> >> > > > > > > > > >> > don't think it's fair
to expect Nick to do it for
> the
> >> > last
> >> > > > > 1.1.x
> >> > > > > > > > > release
> >> > > > > > > > > >> > when it hasn't been done
prior, but I'm not saying
> >> don't
> >> > > do
> >> > > > it
> >> > > > > > if
> >> > > > > > > he
> >> > > > > > > > > >> wants
> >> > > > > > > > > >> > to.
> >> > > > > > > > > >> >
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > > On Dec 4, 2017, at
8:43 AM, Mike Drob <
> >> > mdrob@apache.org
> >> > > >
> >> > > > > > wrote:
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > -1 (non-binding)
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > general:
> >> > > > > > > > > >> > > + sign, sums good
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > src:
> >> > > > > > > > > >> > > - tar missing hbase-native-client
(present in
> tag)
> >> > > > > > > > > >> > > + can build from
source (oracle jdk 1.7.0_80)
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > bin:
> >> > > > > > > > > >> > > - docs/doap_Hbase.rdf
is very out of date
> (possibly
> >> > not
> >> > > > > > actually
> >> > > > > > > > an
> >> > > > > > > > > >> > issue)
> >> > > > > > > > > >> > > - unable to launch
using bin/hbase-start.sh (both
> >> > java 7
> >> > > > and
> >> > > > > > 8),
> >> > > > > > > > got
> >> > > > > > > > > >> > > following error in
.out log file:
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > WARNING! HBase file
layout needs to be upgraded.
> >> You
> >> > > have
> >> > > > > > > version
> >> > > > > > > > > null
> >> > > > > > > > > >> > and
> >> > > > > > > > > >> > > I want version 8.
Consult
> >> > http://hbase.apache.org/book.
> >> > > > html
> >> > > > > > for
> >> > > > > > > > > >> further
> >> > > > > > > > > >> > > information about
upgrading HBase. Is your
> >> > hbase.rootdir
> >> > > > > > valid?
> >> > > > > > > If
> >> > > > > > > > > so,
> >> > > > > > > > > >> > you
> >> > > > > > > > > >> > > may need to run 'hbase
hbck -fixVersionFile'.
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > System Info:
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > mdrob@mdrob-MBP:/tmp/hbase-1.1-0/hbase-1.1.13$
> >> > sw_vers
> >> > > > > > > > > >> > > ProductName: Mac
OS X
> >> > > > > > > > > >> > > ProductVersion: 10.12.6
> >> > > > > > > > > >> > > BuildVersion: 16G1036
> >> > > > > > > > > >> > > mdrob@mdrob-MBP:/tmp/hbase-1.1-0/hbase-1.1.13$
> >> java
> >> > > > > -version
> >> > > > > > > > > >> > > java version "1.7.0_80"
> >> > > > > > > > > >> > > Java(TM) SE Runtime
Environment (build
> >> 1.7.0_80-b15)
> >> > > > > > > > > >> > > Java HotSpot(TM)
64-Bit Server VM (build
> 24.80-b11,
> >> > > mixed
> >> > > > > > mode)
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > Mike
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > >> On Fri, Dec 1,
2017 at 12:31 AM, Nick Dimiduk <
> >> > > > > > > > ndimiduk@apache.org
> >> > > > > > > > > >
> >> > > > > > > > > >> > wrote:
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> I'm happy to
announce the first release
> candidate
> >> of
> >> > > > HBase
> >> > > > > > > 1.1.13
> >> > > > > > > > > >> > >> (HBase-1.1.13RC0)
is available for download at
> >> > > > > > > > > >> > >> *https://dist.apache.org/
> >> > repos/dist/dev/hbase/hbase-1.
> >> > > > > > 1.13RC0/
> >> > > > > > > > > >> > >> <https://dist.apache.org/
> >> > repos/dist/dev/hbase/hbase-1.
> >> > > > > > 1.13RC0/
> >> > > > > > > > >*.
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> This is to be
the final release from branch-1.1.
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> Maven artifacts
are available in the staging
> >> > repository
> >> > > > > > > > > >> > >> *https://repository.apache.org
> >> /content/repositories/
> >> > > > > orgapach
> >> > > > > > > > > >> ehbase-1182
> >> > > > > > > > > >> > >> <https://repository.apache.org
> >> /content/repositories/
> >> > > > > orgapach
> >> > > > > > > > > >> ehbase-1182
> >> > > > > > > > > >> > >*
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> Artifacts are
signed with my code signing subkey
> >> > > > > > > > > 0xAD9039071C3489BD,
> >> > > > > > > > > >> > >> available in
the Apache keys directory
> >> > > > > > > > > >> https://people.apache.org/keys
> >> > > > > > > > > >> > >> /committer/ndimiduk.asc
> >> > > > > > > > > >> > >> <https://people.apache.org/
> >> > keys/committer/ndimiduk.asc
> >> > > >
> >> > > > > and
> >> > > > > > in
> >> > > > > > > > our
> >> > > > > > > > > >> KEYS
> >> > > > > > > > > >> > >> file http://www-us.apache.org/dist/
> >> > > > > > > > > >> > >> hbase/KEYS <http://www-us.apache.org/dist
> >> /hbase/KEYS
> >> > >.
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> There's also
a signed tag for this release at
> >> > > > > > > > > >> > >> *https://git-wip-us.apache.
> >> > > org/repos/asf?p=hbase.git;a=
> >> > > > tag
> >> > > > > ;
> >> > > > > > > > > >> > >> h=16a04e6629e614c7900c443f3a29cdba92dd7b7e
> >> > > > > > > > > >> > >> <https://git-wip-us.apache.
> >> > > org/repos/asf?p=hbase.git;a=
> >> > > > tag
> >> > > > > ;
> >> > > > > > > > > >> > >> h=16a04e6629e614c7900c443f3a29cdba92dd7b7e>*
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> The detailed
source and binary compatibility
> >> report
> >> > vs
> >> > > > > 1.1.12
> >> > > > > > > has
> >> > > > > > > > > >> been
> >> > > > > > > > > >> > >> published for
your review, at
> >> > > > > > > > > >> > >> *https://home.apache.org/~ndim
> >> iduk/1.1.12_1.1.13RC0_
> >> > > > > compat_
> >> > > > > > > > > >> report.html
> >> > > > > > > > > >> > >> <https://home.apache.org/~ndim
> >> iduk/1.1.12_1.1.13RC0_
> >> > > > > compat_
> >> > > > > > > > > >> report.html
> >> > > > > > > > > >> > >*
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> HBase 1.1.13
is the thirteenth and final patch
> >> > release
> >> > > in
> >> > > > > the
> >> > > > > > > > HBase
> >> > > > > > > > > >> 1.1
> >> > > > > > > > > >> > >> line, continuing
on the theme of bringing a
> >> stable,
> >> > > > > reliable
> >> > > > > > > > > >> database to
> >> > > > > > > > > >> > >> the Hadoop and
NoSQL communities. This release
> >> > includes
> >> > > > > over
> >> > > > > > 40
> >> > > > > > > > > >> resolved
> >> > > > > > > > > >> > >> issues since
the 1.1.12 release; the majority of
> >> > these
> >> > > > > > changes
> >> > > > > > > > are
> >> > > > > > > > > to
> >> > > > > > > > > >> > build
> >> > > > > > > > > >> > >> tooling rather
than the product itself. Notable
> >> > product
> >> > > > > > > > correctness
> >> > > > > > > > > >> > fixes
> >> > > > > > > > > >> > >> include HBASE-18665
and HBASE-19052.
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> The full list
of fixes included in this release
> is
> >> > > > > available
> >> > > > > > at
> >> > > > > > > > > >> > >> *https://issues.apache.org/
> >> > > jira/secure/ReleaseNote.jspa?
> >> > > > > proj
> >> > > > > > > > > >> > >> ectId=12310753&version=12341346
> >> > > > > > > > > >> > >> <https://issues.apache.org/
> >> > > jira/secure/ReleaseNote.jspa?
> >> > > > > proj
> >> > > > > > > > > >> > >> ectId=12310753&version=12341346>*
> >> > > > > > > > > >> > >> and
> >> > > > > > > > > >> > >> and in the CHANGES.txt
file included in the
> >> > > distribution.
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> Please try out
this candidate and vote +/-1 by
> >> 23:59
> >> > > > > Pacific
> >> > > > > > > time
> >> > > > > > > > > on
> >> > > > > > > > > >> > >> Friday, 2017-12-08
as to whether we should
> release
> >> > > these
> >> > > > > > > > artifacts
> >> > > > > > > > > as
> >> > > > > > > > > >> > HBase
> >> > > > > > > > > >> > >> 1.1.13.
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> > >> Thanks,
> >> > > > > > > > > >> > >> Nick
> >> > > > > > > > > >> > >>
> >> > > > > > > > > >> >
> >> > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > --
> >> > > > > > > > > > Best regards,
> >> > > > > > > > > > Andrew
> >> > > > > > > > > >
> >> > > > > > > > > > Words like orphans lost among the
crosstalk, meaning
> >> torn
> >> > > from
> >> > > > > > > truth's
> >> > > > > > > > > > decrepit hands
> >> > > > > > > > > >    - A23, Crosstalk
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > Best regards,
> >> > > > > > > > > Andrew
> >> > > > > > > > >
> >> > > > > > > > > Words like orphans lost among the crosstalk,
meaning
> torn
> >> > from
> >> > > > > > truth's
> >> > > > > > > > > decrepit hands
> >> > > > > > > > >    - A23, Crosstalk
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Best regards,
> >> > > > > > > Andrew
> >> > > > > > >
> >> > > > > > > Words like orphans lost among the crosstalk, meaning
torn
> from
> >> > > > truth's
> >> > > > > > > decrepit hands
> >> > > > > > >    - A23, Crosstalk
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Best regards,
> >> > Andrew
> >> >
> >> > Words like orphans lost among the crosstalk, meaning torn from truth's
> >> > decrepit hands
> >> >    - A23, Crosstalk
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message