apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Narsude <che...@datatorrent.com>
Subject Re: License header
Date Wed, 23 Sep 2015 23:33:03 GMT
Yes Ram, this is handled by license plugin already. Do you think we should
hook up the license:check with the pull request?

--
Chetan


On Wed, Sep 23, 2015 at 4:17 PM, Munagala Ramanath <ram@datatorrent.com>
wrote:

> If we have a solution that we are happy with, there is no need to resort to
> my script.
>
> I wrote the script to address what I thought was a one-time need as
> described in the message
> from Thomas. Our files have a lot of quirks in the the way the copyrights
> notices are present,
> for example, I found these variants (among others):
>
>    /*
>     *  Copyright (c) 2012-2013 DataTorrent, Inc.
>     *  All Rights Reserved.
>     */
>
>     /*
>      * Copyright (c) 2014 DataTorrent, Inc. ALL Rights Reserved.
>      */
>
>     /**
>      * Put your copyright and license info here.
>      */
>
> A custom script that recognizes and deals with these variants will likely
> do a better job that
> a standard plugin (for example, we need to preserve the original date, we
> cannot change everything
> to 2015). I'm not suggesting it for regular use, just a one-time
> conversion.
>
> So, to repeat, if we already have a solution we like, please disregard my
> script.
>
> Ram
>
> On Wed, Sep 23, 2015 at 4:06 PM, Chetan Narsude <chetan@datatorrent.com>
> wrote:
>
> > Ram,
> >
> >   It does have a licensed header which violates the generally followed
> > licensing rules. The first line is blank (there is a issue open to be
> > lenient in that case as I remember I saw sometime ago). There are a few
> > more files with the same viloation. Regardless, I am not saying that it's
> > perfect. But why reinvent the wheel? You can work with the plugin writer
> > like 100 others have done and make it better if you feel that it's not
> good
> > enough.
> >
> >  A better way to find all the files which violate the formatting is to
> > format the files which do not follow the rule, I get the following. It
> has
> > a few more files than you mentioned:
> >
> > chetan@chubi:~/work/apex$ mvn license:format -Dlicense.skip=false
> > [INFO] Scanning for projects...
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Reactor Build Order:
> > [INFO]
> > [INFO] Realtime Stream Processing Framework
> > [INFO] DataTorrent API
> > [INFO] Base Library
> > [INFO] Buffer Server
> > [INFO] Streaming Application Manager
> > [INFO] Apex Application Maven archetype
> > [INFO] Apex App Configuration Maven archetype
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building Realtime Stream Processing Framework 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-framework
> > ---
> > [INFO] Updating license headers...
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building DataTorrent API 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-api ---
> > [INFO] Updating license headers...
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/api/src/main/java/com/datatorrent/api/annotation/Name.java
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building Base Library 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-common ---
> > [INFO] Updating license headers...
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/common/src/main/java/com/datatorrent/common/metric/SingleMetricAggregator.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/common/src/main/java/com/datatorrent/common/metric/sum/LongSumAggregator.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/common/src/main/java/com/datatorrent/common/metric/sum/DoubleSumAggregator.java
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building Buffer Server 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @
> dt-bufferserver
> > ---
> > [INFO] Updating license headers...
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/bufferserver/src/main/java/com/datatorrent/bufferserver/util/VarInt.java
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building Streaming Application Manager 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-engine ---
> > [INFO] Updating license headers...
> > [WARNING] Unknown file extension:
> > /home/chetan/work/apex/engine/src/main/scripts/dtcli
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/plan/logical/StreamCodecWrapperForPersistance.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/engine/Slider.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/stream/PartitionAwareSinkForPersistence.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/webapp/asm/FieldSignatureVisitor.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/webapp/asm/CompactAnnotationNode.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/webapp/asm/CompactFieldNode.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/api/AppDataSource.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/util/PubSubWebSocketServlet.java
> > [INFO] Updating license header in:
> >
> >
> /home/chetan/work/apex/engine/src/test/java/com/datatorrent/stram/plan/StreamPersistanceTests.java
> > [WARNING] Unable to find a comment style definition for some files. You
> may
> > want to add a custom mapping for the relevant file extensions.
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building Apex Application Maven archetype 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @
> > apex-app-archetype ---
> > [INFO] Updating license headers...
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building Apex App Configuration Maven archetype 3.2.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO]
> > [INFO] --- license-maven-plugin:2.11:format (default-cli) @
> > apex-conf-archetype ---
> > [INFO] Updating license headers...
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Reactor Summary:
> > [INFO]
> > [INFO] Realtime Stream Processing Framework ............... SUCCESS [
> >  0.889 s]
> > [INFO] DataTorrent API .................................... SUCCESS [
> >  0.162 s]
> > [INFO] Base Library ....................................... SUCCESS [
> >  0.093 s]
> > [INFO] Buffer Server ...................................... SUCCESS [
> >  0.072 s]
> > [INFO] Streaming Application Manager ...................... SUCCESS [
> >  0.408 s]
> > [INFO] Apex Application Maven archetype ................... SUCCESS [
> >  0.233 s]
> > [INFO] Apex App Configuration Maven archetype ............. SUCCESS [
> >  0.013 s]
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Total time: 2.521 s
> > [INFO] Finished at: 2015-09-23T16:00:02-07:00
> > [INFO] Final Memory: 13M/471M
> > [INFO]
> > ------------------------------------------------------------------------
> >
> >
> >
> >
> >
> > On Wed, Sep 23, 2015 at 4:00 PM, Munagala Ramanath <ram@datatorrent.com>
> > wrote:
> >
> > > When I run that command, I get the same error about Name.java but that
> > file
> > > _does_ have a
> > > license header.
> > >
> > > The files identified by my script as having issues are not shown:
> > >
> > > ============================================================
> > > 3 Java files with no comments
> > > 0:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/plan/logical/StreamCodecWrapperForPersistance.java
> > > 1:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/stream/PartitionAwareSinkForPersistence.java
> > > 2:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/test/java/com/datatorrent/stram/plan/StreamPersistanceTests.java
> > > ============================================================
> > > 4 Java files with no matching copyright comment
> > > 0:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java
> > > 1:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java
> > > 2:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java
> > > 3:
> > >
> > >
> >
> /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java
> > >
> > >
> > > Ram
> > >
> > > On Wed, Sep 23, 2015 at 3:46 PM, Thomas Weise <thomas@datatorrent.com>
> > > wrote:
> > >
> > > > Ram,
> > > >
> > > > You need to run
> > > >
> > > > mvn license:check -Dlicense.skip=false
> > > >
> > > > I get the following:
> > > >
> > > > [INFO] --- license-maven-plugin:2.11:check (default-cli) @ dt-api ---
> > > > [INFO] Checking licenses...
> > > > [WARNING] Missing header in:
> > > >
> > > >
> > >
> >
> /home/hdev/devel/dt/Apex/api/src/main/java/com/datatorrent/api/annotation/Name.java
> > > >
> > > > Thomas
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Sep 23, 2015 at 3:34 PM, Munagala Ramanath <
> > ram@datatorrent.com>
> > > > wrote:
> > > >
> > > > > It's in the output.txt file uploaded with my earlier message but
> here
> > > is
> > > > an
> > > > > extract from the end of it:
> > > > >
> > > > > ============================================================
> > > > > 3 Java files with no comments
> > > > > 0:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/plan/logical/StreamCodecWrapperForPersistance.java
> > > > > 1:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/stream/PartitionAwareSinkForPersistence.java
> > > > > 2:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/test/java/com/datatorrent/stram/plan/StreamPersistanceTests.java
> > > > > ============================================================
> > > > > 4 Java files with no matching copyright comment
> > > > > 0:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java
> > > > > 1:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java
> > > > > 2:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java
> > > > > 3:
> > > > >
> > > > >
> > > >
> > >
> >
> /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java
> > > > >
> > > > >
> > > > > On Wed, Sep 23, 2015 at 2:44 PM, Chetan Narsude <
> > > chetan@datatorrent.com>
> > > > > wrote:
> > > > >
> > > > > > which files are those?
> > > > > >
> > > > > > --
> > > > > > chetan
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 23, 2015 at 12:51 PM, Munagala Ramanath <
> > > > ram@datatorrent.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Not sure what it's checking but "*mvn license:check*" shows
me
> no
> > > > list
> > > > > of
> > > > > > > files in core.
> > > > > > > Yet, my script finds at least 3 java files with no comments
at
> > all.
> > > > > > >
> > > > > > > Ram
> > > > > > >
> > > > > > > On Wed, Sep 23, 2015 at 11:54 AM, Chetan Narsude <
> > > > > chetan@datatorrent.com
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > mvn license:check
> > > > > > > > mvn license:format
> > > > > > > >
> > > > > > > > At the top level pom does magic for you.
> > > > > > > >
> > > > > > > > On Wednesday, September 23, 2015, Munagala Ramanath
<
> > > > > > ram@datatorrent.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Thomas and Chris:
> > > > > > > > >
> > > > > > > > > I wrote a Ruby script to automate this process.
Currently,
> it
> > > > makes
> > > > > > no
> > > > > > > > > changes and just
> > > > > > > > > reports what it finds. The script is attached.
> > > > > > > > >
> > > > > > > > > I ran it like this: ruby -w fix-license.rb -f
> > ~/src/apex/apex >
> > > > > > > > output.txt
> > > > > > > > > The output is also attached.
> > > > > > > > >
> > > > > > > > > Let me know if you'd like me to enhance this
to actually
> add
> > > the
> > > > > new
> > > > > > > > > copyright notice to the top of
> > > > > > > > > the files where it does detect some form of copyright
> notice
> > > > (there
> > > > > > > will
> > > > > > > > > still be a few files where
> > > > > > > > > no such notice was found and those will need
manual
> > > > augmentation).
> > > > > > > > >
> > > > > > > > > Ram
> > > > > > > > >
> > > > > > > > > On Tue, Sep 22, 2015 at 9:30 AM, Chris Nauroth
<
> > > > > > > cnauroth@hortonworks.com
> > > > > > > > > <javascript:_e(%7B%7D,'cvml','cnauroth@hortonworks.com
> ');>>
> > > > wrote:
> > > > > > > > >
> > > > > > > > >> Hi Thomas,
> > > > > > > > >>
> > > > > > > > >> I think you are correct.  Here is the official
Apache
> > > statement
> > > > on
> > > > > > > what
> > > > > > > > to
> > > > > > > > >> do with the source headers:
> > > > > > > > >>
> > > > > > > > >> http://www.apache.org/legal/src-headers.html
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> I think your result (the second example below)
matches up
> > with
> > > > > that.
> > > > > > > > >>
> > > > > > > > >> As a reminder, I have a work-in-progress
patch for the
> > > non-Java
> > > > > > files
> > > > > > > in
> > > > > > > > >> Apex here:
> > > > > > > > >>
> > > > > > > > >> https://issues.apache.org/jira/browse/APEXCORE-1
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> It would be nice if we could come up with
a script to chop
> > out
> > > > the
> > > > > > > > >> DataTorrent header and replace it with the
ASF header.
> > Then,
> > > we
> > > > > > could
> > > > > > > > >> simply run that across *.java in both Apex
and Malhar.
> > > > > > > > >>
> > > > > > > > >> --Chris Nauroth
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On 9/22/15, 9:24 AM, "Thomas Weise" <
> thomas@datatorrent.com
> > > > > > > > >> <javascript:_e(%7B%7D,'cvml','thomas@datatorrent.com');>>
> > > > wrote:
> > > > > > > > >>
> > > > > > > > >> >Hit the wrong button...
> > > > > > > > >> >
> > > > > > > > >> >This question is about the license header
for the source
> > > files.
> > > > > > > > Currently
> > > > > > > > >> >we have:
> > > > > > > > >> >
> > > > > > > > >> >/**
> > > > > > > > >> > * Copyright (C) 2015 DataTorrent, Inc.
> > > > > > > > >> > *
> > > > > > > > >> > * Licensed under the Apache License,
Version 2.0 (the
> > > > > "License");
> > > > > > > > >> > * you may not use this file except in
compliance with
> the
> > > > > License.
> > > > > > > > >> > * You may obtain a copy of the License
at
> > > > > > > > >> > *
> > > > > > > > >> > *         http://www.apache.org/licenses/LICENSE-2.0
> > > > > > > > >> > *
> > > > > > > > >> > * Unless required by applicable law
or agreed to in
> > writing,
> > > > > > > software
> > > > > > > > >> > * distributed under the License is distributed
on an "AS
> > IS"
> > > > > > BASIS,
> > > > > > > > >> > * WITHOUT WARRANTIES OR CONDITIONS OF
ANY KIND, either
> > > express
> > > > > or
> > > > > > > > >> >implied.
> > > > > > > > >> > * See the License for the specific language
governing
> > > > > permissions
> > > > > > > and
> > > > > > > > >> > * limitations under the License.
> > > > > > > > >> > */
> > > > > > > > >> >
> > > > > > > > >> >The first line needs to go. And a reference
to NOTICE
> needs
> > > to
> > > > be
> > > > > > > added
> > > > > > > > >> as
> > > > > > > > >> >shown below?
> > > > > > > > >> >
> > > > > > > > >> >/*
> > > > > > > > >> > * Licensed to the Apache Software Foundation
(ASF) under
> > one
> > > > or
> > > > > > more
> > > > > > > > >> > * contributor license agreements.  See
the NOTICE file
> > > > > distributed
> > > > > > > > with
> > > > > > > > >> > * this work for additional information
regarding
> copyright
> > > > > > > ownership.
> > > > > > > > >> > * The ASF licenses this file to You
under the Apache
> > > License,
> > > > > > > Version
> > > > > > > > >> 2.0
> > > > > > > > >> > * (the "License"); you may not use this
file except in
> > > > > compliance
> > > > > > > with
> > > > > > > > >> > * the License.  You may obtain a copy
of the License at
> > > > > > > > >> > *
> > > > > > > > >> > *      http://www.apache.org/licenses/LICENSE-2.0
> > > > > > > > >> > *
> > > > > > > > >> > * Unless required by applicable law
or agreed to in
> > writing,
> > > > > > > software
> > > > > > > > >> > * distributed under the License is distributed
on an "AS
> > IS"
> > > > > > BASIS,
> > > > > > > > >> > * WITHOUT WARRANTIES OR CONDITIONS OF
ANY KIND, either
> > > express
> > > > > or
> > > > > > > > >> >implied.
> > > > > > > > >> > * See the License for the specific language
governing
> > > > > permissions
> > > > > > > and
> > > > > > > > >> > * limitations under the License.
> > > > > > > > >> > */
> > > > > > > > >> >
> > > > > > > > >> >On Tue, Sep 22, 2015 at 9:19 AM, Thomas
Weise <
> > > > > > > thomas@datatorrent.com
> > > > > > > > >> <javascript:_e(%7B%7D,'cvml','thomas@datatorrent.com');>>
> > > > > > > > >> >wrote:
> > > > > > > > >> >
> > > > > > > > >> >> There
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >> >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message