ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: [DISCUSS] Ignite Update Checker
Date Tue, 26 Sep 2017 13:20:00 GMT
Folks,

Can we add version of current node to web request? This way we will better
understand version distribution, what might help us with certain API
update/deprecate decisions
E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0

Vladimir.

On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <dsetrakyan@apache.org>
wrote:

> I think it is safe to assume at this point that everyone is in general
> agreement, since there are no active objections.
>
> I have filed a ticket for the 2.3 release. Let's try to make it happen:
> https://issues.apache.org/jira/browse/IGNITE-6305
>
> D.
>
> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
>
> >
> >
> > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul.asf@evosent.com>
> > wrote:
> >
> >> Yeah, I guess that's doable as well and requires less management effort
> >> than my suggestion. We could use events [1] to store payload data (e.g.
> >> IP,
> >> version, etc.)
> >
> >
> > Yes, we could use events or some other similar API provided by GA.
> >
> >
> >> What the download page CGI developed in? PHP?
> >>
> >
> > To be honest, no clue. I guess someone in the community can figure it
> out:
> > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> >
> >
> >> However, I'm not sure whether storing this data in a 3rd party (Google)
> is
> >> compliant with the ASF policy. I guess it's no biggie, but if there's
> >> doubt
> >> in the PMC, it's better to ask legal@.
> >
> >
> > I am not sure there is anything to ask about. The whole Ignite website is
> > GA enabled, and all we are doing is accessing a standard web page from
> the
> > Ignite web site. The information gathered from GA is available only to
> the
> > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > question.
> >
> > Even ASF website itself uses GA: https://www.apache.org/
> > foundation/policies/privacy.html
> >
> >
> >> I think Cos said it's OK; maybe Roman can pitch in.
> >>
> >
> >  Sure, would be nice to hear from Roman as well.
> >
> >
> >> Cheers.
> >>
> >> [1]
> >> https://developers.google.com/analytics/devguides/collection
> >> /analyticsjs/events
> >>
> >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org>
> >> wrote:
> >>
> >> > Raul,
> >> >
> >> > Could point about Javascript, it will not work because it executes in
> >> the
> >> > browser. This means we need a server-side script, like CGI we are
> using
> >> on
> >> > our download page.
> >> >
> >> > How about this approach. We create something like ignite-version.cgi
> >> script
> >> > which will invoke a call to GA and then return the latest version.
> >> >
> >> > This should work, right?
> >> >
> >> > D.
> >> >
> >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <raul.asf@evosent.com
> >
> >> > wrote:
> >> >
> >> > > Hey Dmitriy and all
> >> > >
> >> > > Also, since we have GA enabled for the website, we can track how
> many
> >> > times
> >> > > > this page was accessed, which will be equal to the number of
> starts.
> >> > This
> >> > > > way, the counter information is tracked and monitored by the
> Ignite
> >> > PMC.
> >> > >
> >> > >
> >> > > Unfortunately this won't work because GA is loaded via Javascript
on
> >> the
> >> > > browser, so Google will never receive the page hit.
> >> > >
> >> > > Given the constraints, the most viable solution is an HTTPS endpoint
> >> > > running on ASF infra that Ignite invokes via a GET or POST request.
> >> The
> >> > > simplest thing is to write each request in a log file, along with
> the
> >> > > timestamp, the version reported by the client, maybe the IP (not
> sure
> >> > about
> >> > > the ASF rules about this concerning privacy, I guess it's OK if you
> >> make
> >> > it
> >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> creates
> >> on
> >> > > first startup or something.
> >> > >
> >> > > This endpoint would need some basic DDoS protection and blacklisting
> >> to
> >> > > prevent data spoofing.
> >> > >
> >> > > If we'll be implementing this endpoint anyway, then there's no point
> >> > > placing another file on Git or elsewhere for reporting the latest
> >> > versions:
> >> > > the endpoint itself can return them.
> >> > >
> >> > > WDYT?
> >> > >
> >> > > Cheers.
> >> > >
> >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >> > dsetrakyan@apache.org>
> >> > > wrote:
> >> > >
> >> > > > Cos, Raul,
> >> > > >
> >> > > > Thanks for the feedback. I completely agree about Maven Central
> >> being a
> >> > > 3rd
> >> > > > party repo (did not think about that initially). All your
> >> suggestions
> >> > > make
> >> > > > sense, but I would like to keep it as simple as possible, and
so
> far
> >> > > > everything suggested required GIT dependencies and extra work.
> >> > > >
> >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> >> > website
> >> > > > which will have only the latest version. Every time a node starts,
> >> it
> >> > > > receives the latest version from the page and suggests that users
> >> > upgrade
> >> > > > if needed.
> >> > > >
> >> > > > Also, since we have GA enabled for the website, we can track
how
> >> many
> >> > > times
> >> > > > this page was accessed, which will be equal to the number of
> starts.
> >> > This
> >> > > > way, the counter information is tracked and monitored by the
> Ignite
> >> > PMC.
> >> > > >
> >> > > > This approach looks pretty innocent to me and everything is kept
> and
> >> > > > managed within Apache.
> >> > > >
> >> > > > Thoughts?
> >> > > >
> >> > > > D.
> >> > > >
> >> > > >
> >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >> cos@apache.org>
> >> > > > wrote:
> >> > > >
> >> > > > > I agree with Raul.
> >> > > > > - providing a ping-back address to a 3rd party might be
frown
> >> upon by
> >> > > > some.
> >> > > > >   And might have a consequences like stats collection about
> users'
> >> > > > >   infrastructure.
> >> > > > > - checking an ASF git-repo is easy and won't download any
binary
> >> > data:
> >> > > > >   everything is clear text and could be easily monitored
by any
> of
> >> > the
> >> > > > > network
> >> > > > >   diagnostic tools, shall it be required. But it involves
a bit
> of
> >> > the
> >> > > > > release
> >> > > > >   discipline.
> >> > > > > - the binary data download in the runtime is my main concern.
> >> This is
> >> > > the
> >> > > > >   vector of MMA. What if someone gains the control over
the
> >> > repository
> >> > > > and
> >> > > > >   replaces the file with some malicious content.
> >> > > > >
> >> > > > > As for the particular mechanism: IIRC Ignite used to make
a call
> >> to
> >> > an
> >> > > > > external script to check upon the atest software version
> available
> >> > for
> >> > > > > download. In the past, the endpoint was running on a 3rd
party
> >> > server,
> >> > > I
> >> > > > > believe the best way would be to put this script on ASF
infra
> and
> >> > have
> >> > > > the
> >> > > > > "update checker" running in a completely controlled environment.
> >> > > > Actually,
> >> > > > > I
> >> > > > > recall we had this very discussion during the Incubation;
I can
> >> > > probably
> >> > > > > dig
> >> > > > > out the corresponding thread.
> >> > > > >
> >> > > > > Thoughts?
> >> > > > >   Cok
> >> > > > >
> >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >> > > > > > Hey guys
> >> > > > > >
> >> > > > > > In my opinion, maven.org is still owned by a third
party
> >> > (Sonatype).
> >> > > > We
> >> > > > > > don't know what kind of data analysis or intelligence
> extraction
> >> > they
> >> > > > > run.
> >> > > > > >
> >> > > > > > If Ignite servers all over the world were hitting maven.org
> >> > > > periodically
> >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype
a clear
> >> > > > indication
> >> > > > > > of who is running an Ignite server.
> >> > > > > >
> >> > > > > > They could reverse lookup the IP address and find out
what
> >> > > corporation
> >> > > > it
> >> > > > > > is.
> >> > > > > >
> >> > > > > > How about having Ignite check the ASF Git directly?
> >> > > > > >
> >> > > > > > We could use the Git ssh API, but that would require
a new
> >> > > dependency,
> >> > > > > > which I advise against.
> >> > > > > >
> >> > > > > > Alternatively, we could have it scrape this HTML for
new Git
> >> tags:
> >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >> > > > > >
> >> > > > > > Another option is to place a txt file in the root of
the
> master
> >> > > branch
> >> > > > > (e.g
> >> > > > > > LATEST), containing a list of the latest GA versions
for each
> >> major
> >> > > > > version
> >> > > > > > line (1.x, 2.x).
> >> > > > > >
> >> > > > > > I would advocate this last option, but it requires
somebody
> >> > > remembering
> >> > > > > to
> >> > > > > > update the file with every release, unless we automate
it
> with a
> >> > > Maven
> >> > > > > > plugin.
> >> > > > > >
> >> > > > > > Hope that helps!
> >> > > > > >
> >> > > > > >
> >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org>
> wrote:
> >> > > > > >
> >> > > > > > I see nothing wrong with this approach.
> >> > > > > >
> >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think?
Is it
> >> good
> >> > > to
> >> > > > > go?
> >> > > > > >
> >> > > > > > —
> >> > > > > > Denis
> >> > > > > >
> >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan
<
> >> > > > dsetrakyan@apache.org
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > Is everyone OK with this approach? Should I file
a ticket on
> >> it?
> >> > > > > > >
> >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan
<
> >> > > > > dsetrakyan@apache.org>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Igniters,
> >> > > > > > >>
> >> > > > > > >> There has been lots of talk of proposals about
various
> usage
> >> > > metrics
> >> > > > > for
> >> > > > > > >> Ignite and nothing came of it. I would like
to resurrect
> the
> >> > topic
> >> > > > and
> >> > > > > > >> propose something very simple and non-intrusive.
> >> > > > > > >>
> >> > > > > > >> 1. Update Checker
> >> > > > > > >> The main purpose of the update checker is
not to collect
> >> > metrics,
> >> > > > but
> >> > > > > to
> >> > > > > > >> notify users about a new version of Ignite
by accessing
> >> > maven.org
> >> > > > and
> >> > > > > > >> getting the version out of the metadata file:
> >> > > > > > >> http://repo2.maven.org/maven2/
> org/apache/ignite/ignite-core/
> >> > > > > > >> maven-metadata.xml
> >> > > > > > >>
> >> > > > > > >> This way we do not send any information anywhere
and, at
> the
> >> > same
> >> > > > > time,
> >> > > > > > >> urge our users to download and start using
the latest
> >> version of
> >> > > > > Ignite.
> >> > > > > > >>
> >> > > > > > >> 2. Startup Counter
> >> > > > > > >> This piece is optional, but we can also get
an insight in
> how
> >> > many
> >> > > > > times
> >> > > > > > a
> >> > > > > > >> certain Ignite release gets started. This
is just a cool
> >> metric
> >> > > for
> >> > > > > the
> >> > > > > > >> community to gauge the project popularity.
You can think of
> >> it
> >> > as
> >> > > > of a
> >> > > > > > page
> >> > > > > > >> visit counter shown on many websites. We can
even decide to
> >> > > display
> >> > > > > this
> >> > > > > > >> counter on the Ignite website as well.
> >> > > > > > >>
> >> > > > > > >> To do this, we can simply add a JAR in maven
for every
> >> release,
> >> > > e.g.
> >> > > > > > >> ignite-start-counter.jar, which will contain
only 1 byte.
> >> Every
> >> > > time
> >> > > > > an
> >> > > > > > >> Ignite node starts, it will download this
JAR in the
> >> background.
> >> > > > Then
> >> > > > > we
> >> > > > > > >> will be able to view the number of the total
downloads for
> >> this
> >> > > JAR
> >> > > > in
> >> > > > > > >> Maven Central, which is essentially the number
of starts of
> >> > Ignite
> >> > > > > nodes.
> >> > > > > > >>
> >> > > > > > >> *Note that neither of the above suggestions
require Ignite
> to
> >> > send
> >> > > > or
> >> > > > > > >> track any user information whatsoever.*
> >> > > > > > >>
> >> > > > > > >> Please reply suggesting weather you are OK
with this
> >> approach.
> >> > > > > > >>
> >> > > > > > >> D.
> >> > > > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message