Mailing-List: contact dev-help@maven.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Maven Developers List" <dev@maven.apache.org>
Received-SPF: pass (nike.apache.org: domain of anders.g.hammar@gmail.com
 designates 209.85.216.50 as permitted sender)
MIME-Version: 1.0
Sender: anders.g.hammar@gmail.com
In-Reply-To: <1347627817.70007.YahooMailNeo@web28904.mail.ir2.yahoo.com>
References: <20120914125724.D3390238897F@eris.apache.org>
	<1347627817.70007.YahooMailNeo@web28904.mail.ir2.yahoo.com>
Date: Fri, 14 Sep 2012 20:04:23 +0200
Message-ID: 
 <CAKDUN1vcH+HU81eAG46MUZifq3xg53Mw_GeVOp6zOVp0PhnZ8g@mail.gmail.com>
Subject: Re: svn commit: r1384747 - in
 /maven/shared/trunk/maven-shared-utils/src:
 main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java
 test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
From: Anders Hammar <anders@hammar.net>
To: Maven Developers List <dev@maven.apache.org>,
 Mark Struberg <struberg@yahoo.de>
Content-Type: text/plain; charset=ISO-8859-1

Yes, "Snapshot" is not good. "PointInTime"?

/Anders
On Fri, Sep 14, 2012 at 3:03 PM, Mark Struberg <struberg@yahoo.de> wrote:
> Hi!
>
> If anyone has a better name for the 'DirectorySnapshotScanner' I would be happy to rename it.
>
>
> What it makes: it takes a snapshot capture of a directory structure and compares it with another snapshot capture of that directory to calculate a diff (files added/removed)
>
> Not sure though if the name 'snapshot' isn't too heavily associated with dependency-snapshots, wdyt?
>
>
> We will need this for the incremental build support.
>
> LieGrue,
> strub
>
>
>
> ----- Original Message -----
>> From: "struberg@apache.org" <struberg@apache.org>
>> To: commits@maven.apache.org
>> Cc:
>> Sent: Friday, September 14, 2012 2:57 PM
>> Subject: svn commit: r1384747 - in /maven/shared/trunk/maven-shared-utils/src: main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
>>
>> Author: struberg
>> Date: Fri Sep 14 12:57:24 2012
>> New Revision: 1384747
>>
>> URL: http://svn.apache.org/viewvc?rev=1384747&view=rev
>> Log:
>> MSHARED-243 add DirectorySnapshotScanner
>>
>> This allows to take snapshot captures of a directory and
>> calculate the 'diff' between those (files added/removed).
>>
>> Added:
>>
>> maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java
>> (with props)
>>
>> maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
>> (with props)
>>
>> Added:
>> maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java
>> URL:
>> http://svn.apache.org/viewvc/maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java?rev=1384747&view=auto
>> ==============================================================================
>> ---
>> maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java
>> (added)
>> +++
>> maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java
>> Fri Sep 14 12:57:24 2012
>> @@ -0,0 +1,139 @@
>> +package org.apache.maven.shared.utils.io;
>> +
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one
>> + * or more contributor license agreements.  See the NOTICE file
>> + * distributed with this work for additional information
>> + * regarding copyright ownership.  The ASF licenses this file
>> + * to you under the Apache License, Version 2.0 (the
>> + * "License"); you may not use this file except in compliance
>> + * with the License.  You may obtain a copy of the License at
>> + *
>> + *  http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing,
>> + * software distributed under the License is distributed on an
>> + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>> + * KIND, either express or implied.  See the License for the
>> + * specific language governing permissions and limitations
>> + * under the License.
>> + */
>> +
>> +import org.apache.maven.shared.utils.CollectionUtils;
>> +
>> +import java.io.File;
>> +import java.util.ArrayList;
>> +import java.util.List;
>> +import java.util.Set;
>> +
>> +/**
>> + * Scan for files in a directory at a given time and reports removed and added
>> files
>> + * between captures.
>> + */
>> +public class DirectorySnapshotScanner
>> +{
>> +    private final static String[] NO_FILES = new String[0];
>> +
>> +    private File rootDirectory;
>> +    private String[] scannedOldFiles;
>> +    private String[] scannedNewFiles = NO_FILES;
>> +    private String[] filesAdded;
>> +    private String[] filesRemoved;
>> +
>> +    public DirectorySnapshotScanner( File rootDirectory )
>> +    {
>> +        this.rootDirectory = rootDirectory;
>> +    }
>> +
>> +    public void capture()
>> +    {
>> +        DirectoryScanner ds = new DirectoryScanner();
>> +        capture( ds );
>> +    }
>> +
>> +    public void capture( DirectoryScanner ds )
>> +    {
>> +        ds.setBasedir( rootDirectory );
>> +        ds.scan();
>> +
>> +        scannedNewFiles = ds.getIncludedFiles();
>> +
>> +        if ( scannedOldFiles != null )
>> +        {
>> +            calculateDiff( scannedOldFiles, scannedNewFiles );
>> +
>> +        }
>> +        else
>> +        {
>> +            filesAdded = NO_FILES;
>> +            filesRemoved = NO_FILES;
>> +        }
>> +
>> +        // this attempts new files is the baseline for the next scan
>> +        scannedOldFiles = scannedNewFiles;
>> +    }
>> +
>> +    /**
>> +     * @return all files which got scanned during the last capture.
>> +     */
>> +    public String[] getScannedFiles()
>> +    {
>> +        return scannedNewFiles;
>> +    }
>> +
>> +    /**
>> +     * @return all files which got detected as being added between 2 capture
>> calls
>> +     */
>> +    public String[] getFilesAdded()
>> +    {
>> +        return filesAdded;
>> +    }
>> +
>> +    /**
>> +     * @return all files which got detected as being removed between 2 capture
>> calls
>> +     */
>> +    public String[] getFilesRemoved()
>> +    {
>> +        return filesRemoved;
>> +    }
>> +
>> +    /**
>> +     * Determine the file differences between the oldFiles and newFiles.
>> +     * This method will not look for a changed in content but sole in the
>> +     * list of files given.
>> +     *
>> +     * The result of the diff can be queried by the methods
>> +     * {@link #getFilesAdded()} and {@link #getFilesRemoved()}
>> +     *
>> +     * @param oldFiles
>> +     * @param newFiles
>> +     */
>> +    public void calculateDiff( String[] oldFiles, String[] newFiles )
>> +    {
>> +        Set<String> oldFileSet = CollectionUtils.arrayAsHashSet( oldFiles
>> );
>> +        Set<String> newFileSet = CollectionUtils.arrayAsHashSet( newFiles
>> );
>> +
>> +        List<String> added = new ArrayList<String>();
>> +        List<String> removed = new ArrayList<String>();
>> +
>> +        for ( String oldFile : oldFileSet )
>> +        {
>> +            if ( !newFileSet.contains( oldFile ) )
>> +            {
>> +                removed.add( oldFile );
>> +            }
>> +        }
>> +
>> +        for ( String newFile : newFileSet )
>> +        {
>> +            if ( !oldFileSet.contains( newFile ) )
>> +            {
>> +                added.add( newFile );
>> +            }
>> +        }
>> +
>> +        filesAdded = added.toArray( new String[ added.size() ] );
>> +        filesRemoved = removed.toArray( new String[ removed.size() ] );
>> +    }
>> +
>> +}
>>
>> Propchange:
>> maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java
>> ------------------------------------------------------------------------------
>>     svn:eol-style = native
>>
>> Added:
>> maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
>> URL:
>> http://svn.apache.org/viewvc/maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java?rev=1384747&view=auto
>> ==============================================================================
>> ---
>> maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
>> (added)
>> +++
>> maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
>> Fri Sep 14 12:57:24 2012
>> @@ -0,0 +1,161 @@
>> +package org.apache.maven.shared.utils.io;
>> +
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one
>> + * or more contributor license agreements.  See the NOTICE file
>> + * distributed with this work for additional information
>> + * regarding copyright ownership.  The ASF licenses this file
>> + * to you under the Apache License, Version 2.0 (the
>> + * "License"); you may not use this file except in compliance
>> + * with the License.  You may obtain a copy of the License at
>> + *
>> + *  http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing,
>> + * software distributed under the License is distributed on an
>> + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>> + * KIND, either express or implied.  See the License for the
>> + * specific language governing permissions and limitations
>> + * under the License.
>> + */
>> +
>> +
>> +import org.apache.maven.shared.utils.testhelpers.FileTestHelper;
>> +import org.junit.Test;
>> +import org.junit.Rule;
>> +import org.junit.Ignore;
>> +import org.junit.Assert;
>> +import org.junit.rules.TemporaryFolder;
>> +
>> +import java.io.File;
>> +import java.io.IOException;
>> +
>> +public class DirectorySnapshotScannerTest
>> +{
>> +
>> +    @Rule
>> +    public TemporaryFolder tempFolder = new TemporaryFolder();
>> +
>> +    private void createTestData() throws IOException
>> +    {
>> +        File rootDir = tempFolder.getRoot();
>> +        File folder1 = new File( rootDir, "folder1" );
>> +        folder1.mkdirs();
>> +
>> +        FileTestHelper.generateTestFile( new File( rootDir,
>> "file1.txt" ), 11 );
>> +        FileTestHelper.generateTestFile( new File( rootDir,
>> "file2.txt" ), 12 );
>> +        FileTestHelper.generateTestFile( new File( rootDir,
>> "file3.dat" ), 13 );
>> +
>> +        FileTestHelper.generateTestFile( new File( folder1,
>> "file4.txt" ), 14 );
>> +        FileTestHelper.generateTestFile( new File( folder1,
>> "file5.dat" ), 15 );
>> +
>> +        File folder2 = new File( folder1, "ignorefolder" );
>> +        folder2.mkdirs();
>> +        FileTestHelper.generateTestFile( new File( folder2,
>> "file7.txt" ), 17 );
>> +
>> +    }
>> +
>> +    private void removeAndAddSomeFiles() throws IOException
>> +    {
>> +        File rootDir = tempFolder.getRoot();
>> +        File file2 = new File( rootDir, "file2.txt" );
>> +        file2.delete();
>> +
>> +        FileTestHelper.generateTestFile( new File( rootDir,
>> "folder1/file9.txt" ), 15 );
>> +
>> +        File folder2 = new File( rootDir, "folder1/ignorefolder" );
>> +        FileUtils.deleteDirectory( folder2 );
>> +    }
>> +
>> +    @Test
>> +    public void testInitialScan() throws Exception
>> +    {
>> +        createTestData();
>> +
>> +        DirectorySnapshotScanner dss = new DirectorySnapshotScanner(
>> tempFolder.getRoot() );
>> +        Assert.assertNotNull( dss );
>> +
>> +        // we take the initial snapshot which should result in an empty diff
>> +        dss.capture();
>> +
>> +        //X TODO define result of the initial scan. Full directory tree or just
>> an empty array?
>> +        String[] addedFiles = dss.getFilesAdded();
>> +        String[] removedFiles = dss.getFilesRemoved();
>> +        Assert.assertNotNull( addedFiles );
>> +        Assert.assertNotNull( removedFiles );
>> +        Assert.assertEquals( 0, addedFiles.length );
>> +        Assert.assertEquals(0, removedFiles.length );
>> +
>> +        // now we change 3 files. add one and remove
>> +        removeAndAddSomeFiles();
>> +
>> +        dss.capture();
>> +
>> +        addedFiles = dss.getFilesAdded();
>> +        removedFiles = dss.getFilesRemoved();
>> +        Assert.assertNotNull( addedFiles );
>> +        Assert.assertNotNull( removedFiles );
>> +        Assert.assertEquals( 1, addedFiles.length );
>> +        Assert.assertEquals(2, removedFiles.length );
>> +
>> +        String[] allFiles = dss.getScannedFiles();
>> +        Assert.assertNotNull( allFiles );
>> +        Assert.assertEquals(5, allFiles.length );
>> +
>> +    }
>> +
>> +
>> +
>> +    @Ignore("Enable this test to run performance checks")
>> +    @Test
>> +    public void performanceTest() throws Exception {
>> +
>> +        File rootFolder = tempFolder.getRoot();
>> +
>> +        // do some warmup
>> +        for ( int i = 1; i < 200; i++ )
>> +        {
>> +            createTestData();
>> +            removeAndAddSomeFiles();
>> +            FileUtils.deleteDirectory( rootFolder );
>> +        }
>> +
>> +        int cycles = 2000;
>> +
>> +        // and now we take the time _without_
>> +        long startTime = System.nanoTime();
>> +        for ( int i = 1; i < cycles; i++ )
>> +        {
>> +            createTestData();
>> +            removeAndAddSomeFiles();
>> +            FileUtils.deleteDirectory( rootFolder );
>> +            rootFolder.mkdir();
>> +        }
>> +        long endTime = System.nanoTime();
>> +
>> +        long durationEmptyRun = endTime - startTime;
>> +        System.out.println( "durationEmptyRun            [ns]: " +
>> durationEmptyRun);
>> +
>> +        startTime = System.nanoTime();
>> +        for ( int i = 1; i < cycles; i++ )
>> +        {
>> +            createTestData();
>> +            DirectorySnapshotScanner dss = new DirectorySnapshotScanner(
>> rootFolder );
>> +            dss.capture();
>> +            removeAndAddSomeFiles();
>> +            dss.capture();
>> +
>> +            FileUtils.deleteDirectory( rootFolder );
>> +            rootFolder.mkdir();
>> +        }
>> +        endTime = System.nanoTime();
>> +
>> +        long durationWithSnapshotScanner = endTime - startTime;
>> +        System.out.println( "durationWithSnapshotScanner [ns]: " +
>> durationWithSnapshotScanner);
>> +
>> +        long dirScannerOverhead = durationWithSnapshotScanner -
>> durationEmptyRun;
>> +
>> +        System.out.println( "Overhead for n cycles [ns]: " +
>> dirScannerOverhead);
>> +    }
>> +
>> +}
>>
>> Propchange:
>> maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java
>> ------------------------------------------------------------------------------
>>     svn:eol-style = native
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org