ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: [DISCUSS] Ignite Update Checker
Date Fri, 29 Sep 2017 01:02:43 GMT
Sorry guys - I neglected the fact that our lists don't permit
attachments. I have put the screenshot to an external server [1]

[1] https://imgur.com/a/p9FJ9

Thank you!
--
  With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.


On Thu, Sep 28, 2017 at 1:37 PM, Denis Magda <dmagda@apache.org> wrote:
> Cos,
>
> The screenshot was not attached. Could you share it some other way (google drive, etc.)?
I’ve never seen any commercial on the site.
>
> —
> Denis
>
>> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik <cos@apache.org> wrote:
>>
>> I don't see an issue with node version either.
>>
>> Just one more, and it might be slightly irrelevant, issue though... I looked at the
Ignite's site and found the following ads and trackers (that are indeed getting disabled by
my browser).
>> Why are googleads or doubleclick are permitted?
>>
>>
>>
>> Thanks,
>>   Cos
>>
>>
>> --
>>   With regards,
>> Konstantin (Cos) Boudnik
>> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>>
>> Disclaimer: Opinions expressed in this email are those of the author, and do not
necessarily represent the views of any company the author might be affiliated with at the
moment of writing.
>>
>> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan <dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>
wrote:
>> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov <vozerov@gridgain.com <mailto:vozerov@gridgain.com>>
>> wrote:
>>
>> > Folks,
>> >
>> > Can we add version of current node to web request? This way we will better
>> > understand version distribution, what might help us with certain API
>> > update/deprecate decisions
>> > E.g. http://ignite.apache.org/latest.cgi&ver=2.2.0 <http://ignite.apache.org/latest.cgi&ver=2.2.0>
>>
>>
>> Vladimir, I personally do not see a problem with that, as sending the
>> current version to the update checker seems very innocent to me. At the
>> same time, it will allow us to examine the usage of each release and make
>> decisions about dropping backward compatibility or spotting bugs for a
>> certain release.
>>
>> Cos, Raul, any thoughts?
>>
>>
>> >
>> >
>> > Vladimir.
>> >
>> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan <dsetrakyan@apache.org
<mailto:dsetrakyan@apache.org>>
>> > wrote:
>> >
>> > > I think it is safe to assume at this point that everyone is in general
>> > > agreement, since there are no active objections.
>> > >
>> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
>> > > https://issues.apache.org/jira/browse/IGNITE-6305 <https://issues.apache.org/jira/browse/IGNITE-6305>
>> > >
>> > > D.
>> > >
>> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>> > dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>
>> > > wrote:
>> > >
>> > > >
>> > > >
>> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul.asf@evosent.com
<mailto:raul.asf@evosent.com>>
>> > > > wrote:
>> > > >
>> > > >> Yeah, I guess that's doable as well and requires less management
>> > effort
>> > > >> than my suggestion. We could use events [1] to store payload data
>> > (e.g.
>> > > >> IP,
>> > > >> version, etc.)
>> > > >
>> > > >
>> > > > Yes, we could use events or some other similar API provided by GA.
>> > > >
>> > > >
>> > > >> What the download page CGI developed in? PHP?
>> > > >>
>> > > >
>> > > > To be honest, no clue. I guess someone in the community can figure
it
>> > > out:
>> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html <https://svn.apache.org/repos/asf/ignite/site/trunk/download.html>
>> > > >
>> > > >
>> > > >> However, I'm not sure whether storing this data in a 3rd party
>> > (Google)
>> > > is
>> > > >> compliant with the ASF policy. I guess it's no biggie, but if
there's
>> > > >> doubt
>> > > >> in the PMC, it's better to ask legal@.
>> > > >
>> > > >
>> > > > I am not sure there is anything to ask about. The whole Ignite website
>> > is
>> > > > GA enabled, and all we are doing is accessing a standard web page
from
>> > > the
>> > > > Ignite web site. The information gathered from GA is available only
to
>> > > the
>> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
>> > > > question.
>> > > >
>> > > > Even ASF website itself uses GA: https://www.apache.org/ <https://www.apache.org/>
>> > > > foundation/policies/privacy.html
>> > > >
>> > > >
>> > > >> I think Cos said it's OK; maybe Roman can pitch in.
>> > > >>
>> > > >
>> > > >  Sure, would be nice to hear from Roman as well.
>> > > >
>> > > >
>> > > >> Cheers.
>> > > >>
>> > > >> [1]
>> > > >> https://developers.google.com/analytics/devguides/collection <https://developers.google.com/analytics/devguides/collection>
>> > > >> /analyticsjs/events
>> > > >>
>> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>> > > >> dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>
>> > > >> wrote:
>> > > >>
>> > > >> > Raul,
>> > > >> >
>> > > >> > Could point about Javascript, it will not work because it
executes
>> > in
>> > > >> the
>> > > >> > browser. This means we need a server-side script, like CGI
we are
>> > > using
>> > > >> on
>> > > >> > our download page.
>> > > >> >
>> > > >> > How about this approach. We create something like ignite-version.cgi
>> > > >> script
>> > > >> > which will invoke a call to GA and then return the latest
version.
>> > > >> >
>> > > >> > This should work, right?
>> > > >> >
>> > > >> > D.
>> > > >> >
>> > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>> > raul.asf@evosent.com <mailto:raul.asf@evosent.com>
>> > > >
>> > > >> > wrote:
>> > > >> >
>> > > >> > > Hey Dmitriy and all
>> > > >> > >
>> > > >> > > Also, since we have GA enabled for the website, we can
track how
>> > > many
>> > > >> > times
>> > > >> > > > this page was accessed, which will be equal to
the number of
>> > > starts.
>> > > >> > This
>> > > >> > > > way, the counter information is tracked and monitored
by the
>> > > Ignite
>> > > >> > PMC.
>> > > >> > >
>> > > >> > >
>> > > >> > > Unfortunately this won't work because GA is loaded via
Javascript
>> > on
>> > > >> the
>> > > >> > > browser, so Google will never receive the page hit.
>> > > >> > >
>> > > >> > > Given the constraints, the most viable solution is an
HTTPS
>> > endpoint
>> > > >> > > running on ASF infra that Ignite invokes via a GET or
POST
>> > request.
>> > > >> The
>> > > >> > > simplest thing is to write each request in a log file,
along with
>> > > the
>> > > >> > > timestamp, the version reported by the client, maybe
the IP (not
>> > > sure
>> > > >> > about
>> > > >> > > the ASF rules about this concerning privacy, I guess
it's OK if
>> > you
>> > > >> make
>> > > >> > it
>> > > >> > > an opt-in) and a unique node identifier, i.e. a UUID
the node
>> > > creates
>> > > >> on
>> > > >> > > first startup or something.
>> > > >> > >
>> > > >> > > This endpoint would need some basic DDoS protection
and
>> > blacklisting
>> > > >> to
>> > > >> > > prevent data spoofing.
>> > > >> > >
>> > > >> > > If we'll be implementing this endpoint anyway, then
there's no
>> > point
>> > > >> > > placing another file on Git or elsewhere for reporting
the latest
>> > > >> > versions:
>> > > >> > > the endpoint itself can return them.
>> > > >> > >
>> > > >> > > WDYT?
>> > > >> > >
>> > > >> > > Cheers.
>> > > >> > >
>> > > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>> > > >> > dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > > Cos, Raul,
>> > > >> > > >
>> > > >> > > > Thanks for the feedback. I completely agree about
Maven Central
>> > > >> being a
>> > > >> > > 3rd
>> > > >> > > > party repo (did not think about that initially).
All your
>> > > >> suggestions
>> > > >> > > make
>> > > >> > > > sense, but I would like to keep it as simple as
possible, and so
>> > > far
>> > > >> > > > everything suggested required GIT dependencies
and extra work.
>> > > >> > > >
>> > > >> > > > How about Yakov's suggestion. We simply add a page
to the Ignite
>> > > >> > website
>> > > >> > > > which will have only the latest version. Every
time a node
>> > starts,
>> > > >> it
>> > > >> > > > receives the latest version from the page and suggests
that
>> > users
>> > > >> > upgrade
>> > > >> > > > if needed.
>> > > >> > > >
>> > > >> > > > Also, since we have GA enabled for the website,
we can track how
>> > > >> many
>> > > >> > > times
>> > > >> > > > this page was accessed, which will be equal to
the number of
>> > > starts.
>> > > >> > This
>> > > >> > > > way, the counter information is tracked and monitored
by the
>> > > Ignite
>> > > >> > PMC.
>> > > >> > > >
>> > > >> > > > This approach looks pretty innocent to me and everything
is kept
>> > > and
>> > > >> > > > managed within Apache.
>> > > >> > > >
>> > > >> > > > Thoughts?
>> > > >> > > >
>> > > >> > > > D.
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik
<
>> > > >> cos@apache.org <mailto:cos@apache.org>>
>> > > >> > > > wrote:
>> > > >> > > >
>> > > >> > > > > I agree with Raul.
>> > > >> > > > > - providing a ping-back address to a 3rd party
might be frown
>> > > >> upon by
>> > > >> > > > some.
>> > > >> > > > >   And might have a consequences like stats
collection about
>> > > users'
>> > > >> > > > >   infrastructure.
>> > > >> > > > > - checking an ASF git-repo is easy and won't
download any
>> > binary
>> > > >> > data:
>> > > >> > > > >   everything is clear text and could be easily
monitored by
>> > any
>> > > of
>> > > >> > the
>> > > >> > > > > network
>> > > >> > > > >   diagnostic tools, shall it be required.
But it involves a
>> > bit
>> > > of
>> > > >> > the
>> > > >> > > > > release
>> > > >> > > > >   discipline.
>> > > >> > > > > - the binary data download in the runtime
is my main concern.
>> > > >> This is
>> > > >> > > the
>> > > >> > > > >   vector of MMA. What if someone gains the
control over the
>> > > >> > repository
>> > > >> > > > and
>> > > >> > > > >   replaces the file with some malicious content.
>> > > >> > > > >
>> > > >> > > > > As for the particular mechanism: IIRC Ignite
used to make a
>> > call
>> > > >> to
>> > > >> > an
>> > > >> > > > > external script to check upon the atest software
version
>> > > available
>> > > >> > for
>> > > >> > > > > download. In the past, the endpoint was running
on a 3rd party
>> > > >> > server,
>> > > >> > > I
>> > > >> > > > > believe the best way would be to put this
script on ASF infra
>> > > and
>> > > >> > have
>> > > >> > > > the
>> > > >> > > > > "update checker" running in a completely controlled
>> > environment.
>> > > >> > > > Actually,
>> > > >> > > > > I
>> > > >> > > > > recall we had this very discussion during
the Incubation; I
>> > can
>> > > >> > > probably
>> > > >> > > > > dig
>> > > >> > > > > out the corresponding thread.
>> > > >> > > > >
>> > > >> > > > > Thoughts?
>> > > >> > > > >   Cok
>> > > >> > > > >
>> > > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani
wrote:
>> > > >> > > > > > Hey guys
>> > > >> > > > > >
>> > > >> > > > > > In my opinion, maven.org <http://maven.org/>
is still owned by a third party
>> > > >> > (Sonatype).
>> > > >> > > > We
>> > > >> > > > > > don't know what kind of data analysis
or intelligence
>> > > extraction
>> > > >> > they
>> > > >> > > > > run.
>> > > >> > > > > >
>> > > >> > > > > > If Ignite servers all over the world
were hitting maven.org <http://maven.org/>
>> > > >> > > > periodically
>> > > >> > > > > > asking for an Ignite Maven artifact,
it gives Sonatype a
>> > clear
>> > > >> > > > indication
>> > > >> > > > > > of who is running an Ignite server.
>> > > >> > > > > >
>> > > >> > > > > > They could reverse lookup the IP address
and find out what
>> > > >> > > corporation
>> > > >> > > > it
>> > > >> > > > > > is.
>> > > >> > > > > >
>> > > >> > > > > > How about having Ignite check the ASF
Git directly?
>> > > >> > > > > >
>> > > >> > > > > > We could use the Git ssh API, but that
would require a new
>> > > >> > > dependency,
>> > > >> > > > > > which I advise against.
>> > > >> > > > > >
>> > > >> > > > > > Alternatively, we could have it scrape
this HTML for new Git
>> > > >> tags:
>> > > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
<https://git-wip-us.apache.org/repos/asf?p=ignite.git>
>> > > >> > > > > >
>> > > >> > > > > > Another option is to place a txt file
in the root of the
>> > > master
>> > > >> > > branch
>> > > >> > > > > (e.g
>> > > >> > > > > > LATEST), containing a list of the latest
GA versions for
>> > each
>> > > >> major
>> > > >> > > > > version
>> > > >> > > > > > line (1.x, 2.x).
>> > > >> > > > > >
>> > > >> > > > > > I would advocate this last option, but
it requires somebody
>> > > >> > > remembering
>> > > >> > > > > to
>> > > >> > > > > > update the file with every release, unless
we automate it
>> > > with a
>> > > >> > > Maven
>> > > >> > > > > > plugin.
>> > > >> > > > > >
>> > > >> > > > > > Hope that helps!
>> > > >> > > > > >
>> > > >> > > > > >
>> > > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
<mailto:dmagda@apache.org>>
>> > > wrote:
>> > > >> > > > > >
>> > > >> > > > > > I see nothing wrong with this approach.
>> > > >> > > > > >
>> > > >> > > > > > Cos, Roman, Raul, as Apache veterans,
what do you think? Is
>> > it
>> > > >> good
>> > > >> > > to
>> > > >> > > > > go?
>> > > >> > > > > >
>> > > >> > > > > > —
>> > > >> > > > > > Denis
>> > > >> > > > > >
>> > > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy
Setrakyan <
>> > > >> > > > dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>
>> > > >> > > > > >
>> > > >> > > > > > wrote:
>> > > >> > > > > > >
>> > > >> > > > > > > Is everyone OK with this approach?
Should I file a ticket
>> > on
>> > > >> it?
>> > > >> > > > > > >
>> > > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM,
Dmitriy Setrakyan <
>> > > >> > > > > dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>
>> > > >> > > > > > > wrote:
>> > > >> > > > > > >
>> > > >> > > > > > >> Igniters,
>> > > >> > > > > > >>
>> > > >> > > > > > >> There has been lots of talk
of proposals about various
>> > > usage
>> > > >> > > metrics
>> > > >> > > > > for
>> > > >> > > > > > >> Ignite and nothing came of it.
I would like to resurrect
>> > > the
>> > > >> > topic
>> > > >> > > > and
>> > > >> > > > > > >> propose something very simple
and non-intrusive.
>> > > >> > > > > > >>
>> > > >> > > > > > >> 1. Update Checker
>> > > >> > > > > > >> The main purpose of the update
checker is not to collect
>> > > >> > metrics,
>> > > >> > > > but
>> > > >> > > > > to
>> > > >> > > > > > >> notify users about a new version
of Ignite by accessing
>> > > >> > maven.org <http://maven.org/>
>> > > >> > > > and
>> > > >> > > > > > >> getting the version out of the
metadata file:
>> > > >> > > > > > >> http://repo2.maven.org/maven2/
<http://repo2.maven.org/maven2/>
>> > > org/apache/ignite/ignite-core/
>> > > >> > > > > > >> maven-metadata.xml
>> > > >> > > > > > >>
>> > > >> > > > > > >> This way we do not send any
information anywhere and, at
>> > > the
>> > > >> > same
>> > > >> > > > > time,
>> > > >> > > > > > >> urge our users to download and
start using the latest
>> > > >> version of
>> > > >> > > > > Ignite.
>> > > >> > > > > > >>
>> > > >> > > > > > >> 2. Startup Counter
>> > > >> > > > > > >> This piece is optional, but
we can also get an insight in
>> > > how
>> > > >> > many
>> > > >> > > > > times
>> > > >> > > > > > a
>> > > >> > > > > > >> certain Ignite release gets
started. This is just a cool
>> > > >> metric
>> > > >> > > for
>> > > >> > > > > the
>> > > >> > > > > > >> community to gauge the project
popularity. You can think
>> > of
>> > > >> it
>> > > >> > as
>> > > >> > > > of a
>> > > >> > > > > > page
>> > > >> > > > > > >> visit counter shown on many
websites. We can even decide
>> > to
>> > > >> > > display
>> > > >> > > > > this
>> > > >> > > > > > >> counter on the Ignite website
as well.
>> > > >> > > > > > >>
>> > > >> > > > > > >> To do this, we can simply add
a JAR in maven for every
>> > > >> release,
>> > > >> > > e.g.
>> > > >> > > > > > >> ignite-start-counter.jar, which
will contain only 1 byte.
>> > > >> Every
>> > > >> > > time
>> > > >> > > > > an
>> > > >> > > > > > >> Ignite node starts, it will
download this JAR in the
>> > > >> background.
>> > > >> > > > Then
>> > > >> > > > > we
>> > > >> > > > > > >> will be able to view the number
of the total downloads
>> > for
>> > > >> this
>> > > >> > > JAR
>> > > >> > > > in
>> > > >> > > > > > >> Maven Central, which is essentially
the number of starts
>> > of
>> > > >> > Ignite
>> > > >> > > > > nodes.
>> > > >> > > > > > >>
>> > > >> > > > > > >> *Note that neither of the above
suggestions require
>> > Ignite
>> > > to
>> > > >> > send
>> > > >> > > > or
>> > > >> > > > > > >> track any user information whatsoever.*
>> > > >> > > > > > >>
>> > > >> > > > > > >> Please reply suggesting weather
you are OK with this
>> > > >> approach.
>> > > >> > > > > > >>
>> > > >> > > > > > >> D.
>> > > >> > > > > > >>
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>

Mime
View raw message