htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: HTrace 4.1 release candidate 2
Date Wed, 24 Feb 2016 21:40:38 GMT
FYI, I just learned that when calling for a vote on general incubator list,
the person calling for the vote can as a convenience, note the existing
binding votes up front so we all don't need to port our votes. Just FYI.
St.Ack

On Wed, Feb 24, 2016 at 1:29 PM, Colin P. McCabe <cmccabe@apache.org> wrote:

> I've posted this on the incubator list.  Don't forget to vote again there
>
> cheers,
> Colin
>
> On Wed, Feb 24, 2016 at 12:26 PM, Colin P. McCabe <cmccabe@apache.org>
> wrote:
> > Thanks, guys.  I will post to the incubator list in a sec.
> >
> > best,
> > Colin
> >
> > On Wed, Feb 24, 2016 at 9:56 AM, Elliott Clark <eclark@apache.org>
> wrote:
> >> +1 on the RC as well.
> >>
> >> Checked the hash.
> >> Created a test app using the api.
> >> Built from src.
> >>
> >>
> >> On Mon, Feb 22, 2016 at 8:37 PM, Stack <stack@duboce.net> wrote:
> >>
> >>> On Mon, Feb 22, 2016 at 3:03 PM, Colin P. McCabe <cmccabe@apache.org>
> >>> wrote:
> >>>
> >>> > There are at least 3 RPC compatibility-breaking changes in
> >>> > htrace-htraced between 4.0.1 and 4.1.0:
> >>> >
> >>> > HTRACE-315 changed the default port for htraced's HTTP interface from
> >>> > 9095 to 9096.
> >>> > HTRACE-237 changes the HTTP wire format slightly for htraced.
> >>> > Previous to this, we just sent a whitespace-separated list of trace
> >>> > spans.  After this, we send an actual JSON object.
> >>> > HTRACE-308 (Deserialize WriteSpans requests incrementally rather than
> >>> > all at once) changes the field "Spans" in the HRPC header to
> >>> > "NumSpans".  It also decreases the maximum RPC size,
> >>> > MAX_HRPC_BODY_LENGTH, from 64 MB to 32 MB.
> >>> >
> >>> >
> >>> Thanks Colin.
> >>>
> >>>
> >>> > To be honest, the main point of 4.0.1 was to stabilize the
> >>> > htrace-core4 API and make a first release with the GUI.  There was
a
> >>> > lot of unfinished business in htraced-- things that we only got to
> >>> > 4.1.  htraced is way more stable in 4.1 since we dealt with things
> >>> > like GC pressure, the client side, and so forth.  And also just
> fixing
> >>> > bugs.  I think we should accept the compatibility break with 4.0.1.
> >>> > However, I agree that it would be nice to support the "old client,
> new
> >>> > server" case.  I filed HTRACE-344 to add a mechanism that makes it
> >>> > easier to detect the case where the client is too new, and give back
> a
> >>> > reasonable response.  I don't think we should adopt a formal
> >>> > compatibility policy-- htraced is not mature enough for that.  But
we
> >>> > can strive to maintain compatibility harder than we have in 4.2 (or
> >>> > whatever the next release ends up being.)
> >>> >
> >>> >
> >>> Ok. Thats good enough I'd say.
> >>>
> >>> +1 on the 4.1 RC.
> >>>
> >>> I checked hash and signature. Built from src and all unit tests passed.
> >>> Started up htraced and put in a few spans with the client.
> >>>
> >>> St.Ack
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> > best,
> >>> > Colin
> >>> >
> >>> >
> >>> > On Mon, Feb 22, 2016 at 10:37 AM, Stack <stack@duboce.net> wrote:
> >>> > > On Mon, Feb 22, 2016 at 9:51 AM, Colin P. McCabe <
> cmccabe@apache.org>
> >>> > wrote:
> >>> > >
> >>> > >> On Sun, Feb 21, 2016 at 8:10 PM, Stack <stack@duboce.net>
wrote:
> >>> > >> > "The rationale for this limitation is that tracing can
simply be
> >>> > disabled
> >>> > >> > for a brief period during the rolling upgrade process."
> >>> > >> >
> >>> > >> > The second time an operator has to do this, they'll just
throw
> away
> >>> > >> tracing
> >>> > >> > as a PITA.
> >>> > >>
> >>> > >> Let's put this in perspective.  Apache Spark recently transitioned
> >>> > >> from Scala 2.10 to 2.11.  Those Scala releases aren't binary
> >>> > >> compatible.  They aren't even source-code compatible, which
means
> that
> >>> > >> people potentially had to rewrite their Spark jobs just to
> perform an
> >>> > >> upgrade... let alone have things continue to work during the
> upgrade.
> >>> > >> And people didn't throw away Spark; it's more popular than
ever.
> >>> > >>
> >>> > >>
> >>> > > By 'perspective', you mean others made a mess so we can too?
> >>> > >
> >>> > >
> >>> > >
> >>> > >> Now: It makes sense for a storage system (particularly a mature
> and
> >>> > >> widely-deployed one) to bend over backwards to stay up during
> >>> > >> upgrades.  That's why HDFS is so strict about this, and HBase
as
> well.
> >>> > >> But they weren't always that strict; we used to break RPC
> >>> > >> compatibility with every release in the earlier days.  Also,
> HTrace is
> >>> > >> not a storage system!  It's a tracing system.  It can be
> unavailable
> >>> > >> for a few hours.  It will be OK.
> >>> > >>
> >>> > >> What's not OK is for us to have CLASSPATH conflicts within
a minor
> >>> > >> version of htrace-core4 that will create problems during rolling
> >>> > >> upgrades.  It is not OK to remove APIs in htrace-core4, and
so
> forth
> >>> > >> and so on.  We thought about this stuff very carefully when
> coming up
> >>> > >> with the compatibility policy to determine what was acceptable
and
> >>> > >> what was not.
> >>> > >>
> >>> > >>
> >>> > > I'm not talking about CLASSPATH, I'm talking about the sending
of
> >>> spans.
> >>> > >
> >>> > > Do you know for sure that a 4.0.1 client can't talk to a
> 4.1.0-based
> >>> > sink?
> >>> > > If so, do you know what is broke?
> >>> > >
> >>> > > Thanks Colin,
> >>> > > St.Ack
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > >> As the project matures, we can think about adopting a more
> generous
> >>> > >> (and much more difficult to implement) compatibility policy
on a
> >>> > >> component-by-component basis.  But currently, a lot of the
> components
> >>> > >> are not very mature.  For example, htrace-flume has seen very
> little
> >>> > >> development.  htrace-htraced is probably the most mature
> component,
> >>> > >> but a lot of its features are new.  4.0.1 was the first release
> with a
> >>> > >> real GUI and usable client support.  Trying to implement an
HDFS
> or
> >>> > >> HBase-style compatibility policy right now would slow down
> development
> >>> > >> greatly, for no gain.
> >>> > >>
> >>> > >> best,
> >>> > >> Colin
> >>> > >>
> >>> > >>
> >>> > >> >
> >>> > >> > Tracing needs to bend to serve the traced systems, not
the
> other way
> >>> > >> around.
> >>> > >> >
> >>> > >> > A 4.0.1 can't talk to a 4.1.0? Do you know how it is
broken?
> >>> > >> >
> >>> > >> > St.Ack
> >>> > >> >
> >>> > >> >
> >>> > >> > On Fri, Feb 19, 2016 at 5:17 PM, Colin P. McCabe <
> >>> cmccabe@apache.org>
> >>> > >> wrote:
> >>> > >> >
> >>> > >> >> Our compatibility policy (see
> >>> > >> >>
> >>> > >> >>
> >>> > >>
> >>> >
> >>>
> http://mail-archives.apache.org/mod_mbox/htrace-dev/201509.mbox/%3C55F8BADC.4050905@oss.nttdata.co.jp%3E
> >>> > >> >> ) only covers the htrace-core4 API right now.  So
we can
> guarantee
> >>> > >> >> that any projects using htrace-core 4.0.1 can upgrade
to
> >>> htrace-core
> >>> > >> >> 4.1.0 without breaking anything.  (This is a more
painful
> guarantee
> >>> > >> >> than it sounds since it means we can't remove functions,
only
> >>> > >> >> deprecate them...  And so forth.)  But it's a very
useful
> guarantee
> >>> > >> >> for our downstream projects.
> >>> > >> >>
> >>> > >> >> However, we don't support mixing and matching versions
of the
> >>> > >> >> SpanReceiver client and server components.  The admin
has to
> roll
> >>> out
> >>> > >> >> a uniform version of those components-- for example,
using
> htraced
> >>> > >> >> 4.0.1 with htrace-htraced.jar 4.1.0 is not supported.
 The
> >>> rationale
> >>> > >> >> for this limitation is that tracing can simply be
disabled for
> a
> >>> > brief
> >>> > >> >> period during the rolling upgrade process.  Also,
the different
> >>> > >> >> SpanReceiver subprojects are at different levels
of maturity,
> and
> >>> > >> >> imposing heavy compatibility guarantees would slow
down
> development
> >>> > >> >> for no real gain.
> >>> > >> >>
> >>> > >> >> best,
> >>> > >> >> Colin
> >>> > >> >>
> >>> > >> >>
> >>> > >> >> On Fri, Feb 19, 2016 at 4:32 PM, Stack <stack@duboce.net>
> wrote:
> >>> > >> >> > Can a 4.0.1 client talk to a 4.1.0 htrace? Has
it been
> tested?
> >>> > >> >> > St.Ack
> >>> > >> >> >
> >>> > >> >> > On Tue, Feb 9, 2016 at 7:00 PM, Colin P. McCabe
<
> >>> > cmccabe@apache.org>
> >>> > >> >> wrote:
> >>> > >> >> >
> >>> > >> >> >> Hi all,
> >>> > >> >> >>
> >>> > >> >> >> I've posted the second release candidate
for HTrace 4.1
> here:
> >>> > >> >> >>
> >>> > >> >> >>
> http://people.apache.org/~cmccabe/htrace/releases/4.1.0/rc2/
> >>> > >> >> >>
> >>> > >> >> >> The jars have been staged here:
> >>> > >> >> >>
> >>> > >> >> >>
> >>> > >>
> >>>
> https://repository.apache.org/content/repositories/orgapachehtrace-1022
> >>> > >> >> >>
> >>> > >> >> >> Compared to rc1, this rc includes HTRACE-334
and HTRACE-342.
> >>> > >> >> >>
> >>> > >> >> >> HTrace 4.1 brings a lot of robustness improvements.
 There
> were
> >>> > major
> >>> > >> >> >> improvements to htraced and the web UI,
as well as new
> metrics
> >>> > added.
> >>> > >> >> >> There were numerous build fixups, and we
added Docker
> support,
> >>> to
> >>> > >> >> >> ensure a repeatable build.
> >>> > >> >> >>
> >>> > >> >> >> Check it out.  The vote will run for 5 days.
> >>> > >> >> >>
> >>> > >> >> >> cheers,
> >>> > >> >> >> Colin
> >>> > >> >> >>
> >>> > >> >> >>
> >>> > >> >> >> Release Notes - HTrace - Version 4.1
> >>> > >> >> >> ** Bug
> >>> > >> >> >>     * [HTRACE-114] - Fix compilation error
of htrace-hbase
> >>> against
> >>> > >> >> >> hbase-1.0.0
> >>> > >> >> >>     * [HTRACE-238] - Change maven compiler
source level to
> 1.7
> >>> to
> >>> > >> >> >> match targetJdk
> >>> > >> >> >>     * [HTRACE-243] - Remove duplicate maven-assembly-plugin
> >>> > >> >> >> configuration section in htrace-htraced/pom.xml
> >>> > >> >> >>     * [HTRACE-245] - NOTICE.txt: change
"developed by The
> Apache
> >>> > >> >> >> Software...” to "developed at The Apache
Software...”
> >>> > >> >> >>     * [HTRACE-246] - HTrace WebApp not properly
defined and
> >>> > therefore
> >>> > >> >> >> not packaged into .war
> >>> > >> >> >>     * [HTRACE-248] - HTraced should gracefully
shutdown if
> >>> stopped
> >>> > >> >> >>     * [HTRACE-249] - Script and doc on how
to publish
> website
> >>> > >> >> >>     * [HTRACE-251] - Fix "mvn clean" target
> >>> > >> >> >>     * [HTRACE-253] - Tracer loadSamplers
and
> loadSpanReceivers
> >>> > logs
> >>> > >> >> >> are too chatty
> >>> > >> >> >>     * [HTRACE-256] - Change the artifactId
for htrace-core
> in
> >>> > branch
> >>> > >> >> >> 4.0 to be htrace-core4
> >>> > >> >> >>     * [HTRACE-257] - htrace-htraced: add
web symlink rather
> than
> >>> > >> >> >> generating programmatically
> >>> > >> >> >>     * [HTRACE-262] - Temporarily suppress
doclint for Java
> 8 to
> >>> > >> >> >> prevent build failure
> >>> > >> >> >>     * [HTRACE-266] - Make the
> >>> CLIENT_REST_MAX_SPANS_AT_A_TIME_KEY
> >>> > >> >> >> config key more consistent with other configs
> >>> > >> >> >>     * [HTRACE-267] - Move owl logo licensing
information
> from
> >>> > NOTICE
> >>> > >> to
> >>> > >> >> >> LICENSE
> >>> > >> >> >>     * [HTRACE-268] - Remove Units and go-codec
from LICENSE
> >>> since
> >>> > >> they
> >>> > >> >> >> are not contained in the source release
> >>> > >> >> >>     * [HTRACE-272] - TracerPool must not
load multiple
> inscance
> >>> of
> >>> > >> >> >> same receiver class when a simple classname
is given
> >>> > >> >> >>     * [HTRACE-279] - Fix issues where the
> HTracedSpanReceiver
> >>> was
> >>> > >> >> >> using the wrong JSON serialization for spans
and add
> validation
> >>> to
> >>> > >> >> >> htraced REST ingest path
> >>> > >> >> >>     * [HTRACE-280] - htraced: add metrics
about total spans
> >>> added
> >>> > and
> >>> > >> >> >> dropped per address
> >>> > >> >> >>     * [HTRACE-281] - htraced: add example/htraced-conf.xml
> >>> > >> >> >>     * [HTRACE-282] - htraced: reap spans
which are older
> than a
> >>> > >> >> >> configurable interval
> >>> > >> >> >>     * [HTRACE-283] - Heartbeater should
wait for goroutine
> to
> >>> > finish
> >>> > >> on
> >>> > >> >> >> close
> >>> > >> >> >>     * [HTRACE-284] - htrace-htraced, htrace-flume:
do not
> treat
> >>> > the
> >>> > >> >> >> shaded version of commons-logging as provided
> >>> > >> >> >>     * [HTRACE-285] - htraced tool: fix query
parsing and add
> >>> > >> query_test
> >>> > >> >> >>     * [HTRACE-289] - Fix TraceEnabled, etc.
logger methods
> for
> >>> > >> >> >> conditional logging
> >>> > >> >> >>     * [HTRACE-294] - htraced: fix some metrics
issues
> >>> > >> >> >>     * [HTRACE-297] - htraced: avoid serializing
spans to
> json
> >>> > unless
> >>> > >> >> >> TRACE logging is enabled
> >>> > >> >> >>     * [HTRACE-300] - Reaper should be initialized
before
> shards
> >>> > are
> >>> > >> >> >> activated
> >>> > >> >> >>     * [HTRACE-301] - htraced: fix unit tests
that aren't
> waiting
> >>> > for
> >>> > >> >> >> spans to be written, use semaphore for WrittenSpans
> >>> > >> >> >>     * [HTRACE-302] - htraced: Add admissions
control to
> HRPC to
> >>> > limit
> >>> > >> >> >> the number of incoming messages
> >>> > >> >> >>     * [HTRACE-304] - htraced: fix bug with
GREATER_THAN
> queries
> >>> > >> >> >>     * [HTRACE-307] - htraced: queries sometimes
return no
> >>> results
> >>> > >> even
> >>> > >> >> >> when many results exist due to confusion
in iterator usage
> >>> > >> >> >>     * [HTRACE-311] - htraced: Fix logging
to stdout via
> >>> > -Dlog.path=
> >>> > >> >> >>     * [HTRACE-316] - htrace-web: span.js
issue: span ID
> string
> >>> > length
> >>> > >> >> >> is 32, not 36
> >>> > >> >> >>     * [HTRACE-317] - Fix the documentation
for adding
> tracing to
> >>> > an
> >>> > >> >> >> application to reflect HTrace 4.x API changes
> >>> > >> >> >>     * [HTRACE-328] - htraced continues scanning
in some
> cases
> >>> even
> >>> > >> >> >> when no more results are possible
> >>> > >> >> >>
> >>> > >> >> >> ** Improvement
> >>> > >> >> >>     * [HTRACE-342] - centralize building
instructions in
> >>> > BUILDING.txt
> >>> > >> >> >>     * [HTRACE-334] - htrace-web: Make limit
of search and
> >>> children
> >>> > >> API
> >>> > >> >> >> configurable
> >>> > >> >> >>     * [HTRACE-129] - htraced: add /server/stats
REST
> endpoint
> >>> > >> >> >>     * [HTRACE-156] - HTrace GUI: add about
view
> >>> > >> >> >>     * [HTRACE-181] - gui: Split "about"
screen
> >>> > >> >> >>     * [HTRACE-237] - Optimize htraced span
receiver
> >>> > >> >> >>     * [HTRACE-239] - Add
> htrace/impl/TestZipkinSpanReceiver.java
> >>> > >> >> >>     * [HTRACE-260] - htrace-zipkin should
not set the
> obsolete
> >>> > >> >> >> duration field in thrift
> >>> > >> >> >>     * [HTRACE-271] - Add log4j.properties
to all submodule
> tests
> >>> > >> >> >>     * [HTRACE-276] - Shade classes into
> org.apache.htrace.shaded
> >>> > >> >> >> rather than org.apache.htrace
> >>> > >> >> >>     * [HTRACE-286] - htraced: improvements
to logging,
> daemon
> >>> > >> startup,
> >>> > >> >> >> and configuration
> >>> > >> >> >>     * [HTRACE-290] - htraced: Fix per-faculty
log level
> settings
> >>> > and
> >>> > >> >> >> add unit tests for conditional logging
> >>> > >> >> >>     * [HTRACE-291] - rename bin/htrace to
bin/htracedTool
> >>> > >> >> >>     * [HTRACE-292] - "htracedTool version"
should display
> the
> >>> git
> >>> > >> >> >> hash, and -Dgit.version option should be
available for build
> >>> > >> >> >>     * [HTRACE-295] - htraced: setting span.expiry.ms
to 0
> >>> should
> >>> > >> >> >> disable span expiry
> >>> > >> >> >>     * [HTRACE-296] - htraced tests: make
sure local
> settings for
> >>> > >> >> >> HTRACED_WEB_DIR and HTRACE_CONF_DIR don't
affect unit tests
> >>> > >> >> >>     * [HTRACE-298] - htraced: improve datastore
> serialization
> >>> and
> >>> > >> >> metrics
> >>> > >> >> >>     * [HTRACE-303] - Add client-side htraceDropped
log file
> to
> >>> > track
> >>> > >> >> >> dropped spans
> >>> > >> >> >>     * [HTRACE-305] - htrace-web: Use greater-than-or-equal
> >>> rather
> >>> > >> than
> >>> > >> >> >> greater-than in more places
> >>> > >> >> >>     * [HTRACE-306] - htraced: logs should
use UTC
> >>> > >> >> >>     * [HTRACE-308] - Deserialize WriteSpans
requests
> >>> incrementally
> >>> > >> >> >> rather than all at once to optimize GC
> >>> > >> >> >>     * [HTRACE-310] - htracedTool: when there
is an error
> >>> response,
> >>> > >> >> >> print the body of the response
> >>> > >> >> >>     * [HTRACE-312] - htraced: if GOMAXPROCS
is left at 1,
> set it
> >>> > to
> >>> > >> >> >> the number of CPUs
> >>> > >> >> >>     * [HTRACE-313] - htraced span receiver
clientDropped
> file
> >>> > should
> >>> > >> >> >> include timestamps
> >>> > >> >> >>     * [HTRACE-314] - htraced: make datastore
loading safer
> >>> > >> >> >>     * [HTRACE-327] - HTRACE-327: improve
htraced
> command-line
> >>> > parsing
> >>> > >> >> >> and add version command
> >>> > >> >> >>     * [HTRACE-334] - htrace-web: Make limit
of search and
> >>> children
> >>> > >> API
> >>> > >> >> >> configurable
> >>> > >> >> >>     * [HTRACE-335] - htrace-web: Adjust
size of span widget
> >>> > >> >> >>     * [HTRACE-339] - Major type in htrace-flume
README
> >>> > >> >> >>
> >>> > >> >> >> ** New Feature
> >>> > >> >> >>     * [HTRACE-235] - htrace-zipkin - add
Kafka transport
> support
> >>> > >> >> >>     * [HTRACE-277] - htraced: Add /server/conf
endpoint to
> get
> >>> > server
> >>> > >> >> >> configuration
> >>> > >> >> >>     * [HTRACE-278] - htraced: dump thread
stacks and GC
> >>> statistics
> >>> > >> >> >> when SIGQUIT is sent
> >>> > >> >> >>     * [HTRACE-288] - htraced: Add a user
interface to view
> >>> server
> >>> > >> >> >> version, metrics, and configuration
> >>> > >> >> >>     * [HTRACE-293] - htrace-web: control-click
should fully
> >>> expand
> >>> > >> trace
> >>> > >> >> >> trees
> >>> > >> >> >>     * [HTRACE-299] - htraced: add /server/debugInfo
REST
> >>> endpoint
> >>> > to
> >>> > >> >> >> get stack traces and GC stats
> >>> > >> >> >>     * [HTRACE-309] - htraced: improve leveldb
configuration
> >>> > >> >> >>     * [HTRACE-323] - htrace-web: change
the cursor to a
> spinner
> >>> > while
> >>> > >> >> >> a search is in progress
> >>> > >> >> >>     * [HTRACE-332] - htraced: optionally
enable leveldb LRU
> >>> cache
> >>> > >> >> >>
> >>> > >> >> >> ** Task
> >>> > >> >> >>     * [HTRACE-241] - Docker image for HTrace
> >>> > >> >> >>     * [HTRACE-315] - htraced: change default
web port from
> 9095
> >>> to
> >>> > >> 9096
> >>> > >> >> >>     * [HTRACE-319] - mark versions 4.0 and
4.0.1 as released
> >>> > >> >> >>     * [HTRACE-331] - create git tags for
4.0 and 4.0.1
> releases
> >>> > >> >> >>
> >>> > >> >> >> ** Wish
> >>> > >> >> >>     * [HTRACE-269] - HTraceConfiguration
support to get the
> map
> >>> of
> >>> > >> >> >> configurations
> >>> > >> >> >>
> >>> > >> >>
> >>> > >>
> >>> >
> >>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message