commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 35126] New: - [vfs][PATCH] Default External File Monitor
Date Mon, 30 May 2005 18:53:13 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=35126>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=35126

           Summary: [vfs][PATCH] Default External File Monitor
           Product: Commons
           Version: unspecified
          Platform: Other
        OS/Version: other
            Status: NEW
          Severity: normal
          Priority: P2
         Component: VFS
        AssignedTo: commons-dev@jakarta.apache.org
        ReportedBy: xknight@users.sourceforge.net


Did some code cleanup and fixed a bug
1) Merged the addStack and deleteStack into a ModifiedQueue object. In case the
order of adding and removing file objects from the monitored map matters in the
future.
2) Added a call to close a file object before it fires a create event. Fixes
following bug: File created in file system, file create event fired, file
deleted from file system, file delete event fired, same file (name, date/time
modified, file path) created again (copied from another location for example) no
event fired.
3) Reset the parent's children list on queue of a file delete instead of on
removal from map. There should be a slimmer chance of a missed file creation (or
re-creation) because of this.



/*
 * Copyright 2002-2005 The Apache Software Foundation.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.commons.vfs.impl;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.commons.vfs.FileListener;
import org.apache.commons.vfs.FileMonitor;
import org.apache.commons.vfs.FileName;
import org.apache.commons.vfs.FileObject;
import org.apache.commons.vfs.FileSystemException;
import org.apache.commons.vfs.FileType;
import org.apache.commons.vfs.provider.AbstractFileSystem;

import java.util.HashMap;
import java.util.Map;
import java.util.Stack;
import java.util.ArrayList;

/**
 * A polling {@link FileMonitor} implementation.<br />
 * <br />
 * The DefaultFileMonitor is a Thread based polling file system monitor with a 1
 * second delay.<br />
 * <br />
 * <b>Design:</b>
 * <p/>
 * There is a Map of monitors known as FileMonitorAgents. With the thread running,
 * each FileMonitorAgent object is asked to "check" on the file it is
 * responsible for.
 * To do this check, the cache is cleared.
 * </p>
 * <ul>
 * <li>If the file existed before the refresh and it no longer exists, a delete
 * event is fired.</li>
 * <li>If the file existed before the refresh and it still exists, check the
 * last modified timestamp to see if that has changed.</li>
 * <li>If it has, fire a change event.</li>
 * </ul>
 * <p/>
 * With each file delete, the FileMonitorAgent of the parent is asked to
 * re-build its
 * list of children, so that they can be accurately checked when there are new
 * children.<br/>
 * New files are detected during each "check" as each file does a check for new
 * children.
 * If new children are found, create events are fired recursively if recursive
 * descent is
 * enabled.
 * </p>
 * <p/>
 * For performance reasons, added a delay that increases as the number of files
 * monitored
 * increases. The default is a delay of 1 second for every 1000 files processed.
 * </p>
 * <p/>
 * <br /><b>Example usage:</b><pre>
 * FileSystemManager fsManager = VFS.getManager();
 * FileObject listendir = fsManager.resolveFile("/home/username/monitored/");
 * <p/>
 * DefaultFileMonitor fm = new DefaultFileMonitor(new CustomFileListener());
 * fm.setRecursive(true);
 * fm.addFile(listendir);
 * fm.start();
 * </pre>
 * <i>(where CustomFileListener is a class that implements the FileListener
 * interface.)</i>
 *
 * @author <a href="mailto:xknight@users.sourceforge.net">Christopher Ottley</a>
 * @version $Revision: 170205 $ $Date: 2005-05-15 08:21:59Z $
 */
public class DefaultFileMonitor implements Runnable, FileMonitor
{
    private final static Log log = LogFactory.getLog(DefaultFileMonitor.class);

    /**
     * Map from FileName to FileObject being monitored.
     */
    private final Map monitorMap = new HashMap();

    /**
     * The low priority thread used for checking the files being monitored.
     */
    private Thread monitorThread;
   
    /**
     * File objects to be modify the monitor map (added to / deleted from the map) 
     */
    private ModificationQueue mQueue = new ModificationQueue();

    /**
     * A flag used to determine if the monitor thread should be running.
     */
    private boolean shouldRun = true;

    /**
     * A flag used to determine if adding files to be monitored should be recursive.
     */
    private boolean recursive = false;

    /**
     * Set the delay between checks
     */
    private long delay = 1000;

    /**
     * Set the number of files to check until a delay will be inserted
     */
    private int checksPerRun = 1000;

    /**
     * A listener object that if set, is notified on file creation and deletion.
     */
    private final FileListener listener;

    public DefaultFileMonitor(final FileListener listener)
    {
        this.listener = listener;
    }

    /**
     * Access method to get the recursive setting when adding files for monitoring.
     */
    public boolean isRecursive()
    {
        return this.recursive;
    }

    /**
     * Access method to set the recursive setting when adding files for monitoring.
     */
    public void setRecursive(final boolean newRecursive)
    {
        this.recursive = newRecursive;
    }

    /**
     * Access method to get the current FileListener object notified when there
     * are changes with the files added.
     */
    FileListener getFileListener()
    {
        return this.listener;
    }

    /**
     * Adds a file to be monitored.
     */
    public void addFile(final FileObject file)
    {
        synchronized (this.monitorMap)
        {
            if (this.monitorMap.get(file.getName()) == null)
            {
                this.monitorMap.put(file.getName(), new FileMonitorAgent(this,
                        file));

                try
                {
                    if (this.listener != null)
                    {
                        file.getFileSystem().addListener(file, this.listener);
                    }

                    if (file.getType().hasChildren() && this.recursive)
                    {
                        // Traverse the children
                        final FileObject[] children = file.getChildren();
                        for (int i = 0; i < children.length; i++)
                        {
                            this.addFile(children[i]); // Add depth first
                        }
                    }

                }
                catch (FileSystemException fse)
                {
                    log.error(fse.getLocalizedMessage(), fse);
                }

            }
        }
    }

    /**
     * Queues a file for addition to be monitored.
     */
    protected void queueAddFile(final FileObject file)
    {
        this.mQueue.join(new ChangeRec(ChangeRec.ADD, file));
    }
        
    /**
     * Removes a file from being monitored.
     */
    public void removeFile(final FileObject file)
    {
        synchronized (this.monitorMap)
        {
            FileName fn = file.getName();
            if (this.monitorMap.get(fn) != null)
            {
                this.monitorMap.remove(fn);
            }
        }
    }

    /**
     * Queues a file for removal from being monitored.
     * Resets the children list on queue instead on processing of removal
     */
    protected void queueRemoveFile(final FileObject file)
    {
        synchronized (this.monitorMap)
        {
            FileObject parent;
            try
            {
                parent = file.getParent();
            }
            catch (FileSystemException fse)
            {
                parent = null;
            }
    
            if (parent != null)
            { // Not the root
                FileMonitorAgent parentAgent =
                    (FileMonitorAgent) this.monitorMap.get(parent.getName());
                if (parentAgent != null)
                {
                    parentAgent.resetChildrenList();
                }
            }
        }
        
       this.mQueue.join(new ChangeRec(ChangeRec.DELETE, file));
          
    }

    /**
     * Get the delay between runs
     */
    public long getDelay()
    {
        return delay;
    }

    /**
     * Set the delay between runs
     */
    public void setDelay(long delay)
    {
        if (delay > 0)
        {
            this.delay = delay;
        }
        else
        {
            this.delay = 1000;
        }
    }

    /**
     * get the number of files to check per run
     */
    public int getChecksPerRun()
    {
        return checksPerRun;
    }

    /**
     * set the number of files to check per run.
     * a additional delay will be added if there are more files to check
     *  
     * @param checksPerRun a value less than 1 will disable this feature
     */
    public void setChecksPerRun(int checksPerRun)
    {
        this.checksPerRun = checksPerRun;
    }


    /**
     * Starts monitoring the files that have been added.
     */
    public void start()
    {
        if (this.monitorThread == null)
        {
            this.monitorThread = new Thread(this);
            this.monitorThread.setDaemon(true);
            this.monitorThread.setPriority(Thread.MIN_PRIORITY);
        }
        this.monitorThread.start();
    }

    /**
     * Stops monitoring the files that have been added.
     */
    public void stop()
    {
        this.shouldRun = false;
    }

    /**
     * Asks the agent for each file being monitored to check its file for changes.
     */
    public void run()
    {
        mainloop:
        while (!Thread.currentThread().isInterrupted() && this.shouldRun)
        {
            while (!this.mQueue.isEmpty())
            {
                ChangeRec currRec = (ChangeRec)this.mQueue.leave();
                
                if (currRec.type() == ChangeRec.DELETE) 
                {
                    this.removeFile((FileObject) currRec.file());
                }
                else if (currRec.type() == ChangeRec.ADD) 
                {
                    this.addFile((FileObject) currRec.file());
                }
            }

            // For each entry in the map
            Object fileNames[];
            synchronized (this.monitorMap)
            {
                fileNames = this.monitorMap.keySet().toArray();
            }
            for (int iterFileNames = 0; iterFileNames < fileNames.length;
                 iterFileNames++)
            {
                FileName fileName = (FileName) fileNames[iterFileNames];
                FileMonitorAgent agent;
                synchronized (this.monitorMap)
                {
                    agent = (FileMonitorAgent) this.monitorMap.get(fileName);
                }
                if (agent != null)
                {
                    agent.check();
                }

                if (getChecksPerRun() > 0)
                {
                    if ((iterFileNames % getChecksPerRun()) == 0)
                    {
                        try
                        {
                            Thread.sleep(getDelay());
                        }
                        catch (InterruptedException e)
                        {

                        }
                    }
                }

                if (Thread.currentThread().isInterrupted() || !this.shouldRun)
                {
                    continue mainloop;
                }
            }

            try
            {
                Thread.sleep(getDelay());
            }
            catch (InterruptedException e)
            {
                continue;
            }
        }

        this.shouldRun = true;
    }

    /** 
     * A queue holding modifications that are to be done to the map.
     */
    private static class ModificationQueue {
    
        ArrayList store = new ArrayList();
      
        public synchronized boolean join(Object o)
        {
            return store.add(o);
        }
      
        public synchronized Object leave()
        {
            Object result = null;
            
            try
            {
                result = store.remove(0);
            }
            catch (Exception e)
            {
                result = null;
            }
            
            return result;
        }
      
        public synchronized int size()
        {
            return store.size();
        }
      
        public synchronized boolean isEmpty()
        {
            return store.isEmpty();
        }
      
        public synchronized Object peek()
        {
            Object result = null;
            
            try
            {
                result = store.get(0);
            }
            catch (Exception e)
            {
                result = null;
            }
            
            return result;
        }
      
        public synchronized boolean containsKey(String key, KeyRetriever kr)
        {
            boolean result = false;
            for (int i = 0; i < store.size(); i++)
            {
                if (kr.getKey(store.get(i)).equals(key))
                {
                    result = true;
                    break;
                }
            }
            return result;
        }
               
        public synchronized void clear()
        {
            store.clear();
        }
    
    }
    
    
    private static class KeyRetriever {
    
        public String getKey(Object o)
        {
            return ((ChangeRec)o).location();
        }
        
    }
    
    /**
    * Record holding an actionable record.
    */
    private static class ChangeRec {
    
      /** A delete action is to be performed. */
      public static final int DELETE = 0;
    
      /** An add action is to be performed. */
      public static final int ADD = 1;
      
      /** The type of action to perform for the document. */
      private int type = -1;
    
      /** A unique string identifier of object to be deleted. */
      private String location;
      
      /** The file that's changing. */
      private FileObject file;
    
      public ChangeRec(int ptype, FileObject pfile)
      {
        this.type = ptype;
        this.file = pfile;  
        this.location = this.file.getName().toString();
      }
    
      public final String location()
      {
          return this.location;
      }
    
      public final int type()
      {
          return this.type;
      }
      
      public final FileObject file()
      {
          return this.file;
      }
      
    }
    

    /**
     * File monitor agent.
     */
    private static class FileMonitorAgent
    {
        private final FileObject file;
        private final DefaultFileMonitor fm;

        private boolean exists;
        private long timestamp;
        private Map children = null;

        private FileMonitorAgent(DefaultFileMonitor fm, FileObject file)
        {
            this.fm = fm;
            this.file = file;

            this.refresh();
            this.resetChildrenList();

            try
            {
                this.exists = this.file.exists();
            }
            catch (FileSystemException fse)
            {
                this.exists = false;
            }

            try
            {
                this.timestamp = this.file.getContent().getLastModifiedTime();
            }
            catch (FileSystemException fse)
            {
                this.timestamp = -1;
            }

        }

        private void resetChildrenList()
        {
            try
            {
                this.refresh();
                
                if (this.file.getType() == FileType.FOLDER)
                {
                    this.children = new HashMap();
                    FileObject[] childrenList = this.file.getChildren();
                    for (int i = 0; i < childrenList.length; i++)
                    {
                        this.children.put(childrenList[i].getName(), new
                            Object()); // null?
                    }
                }
            }
            catch (FileSystemException fse)
            {
                this.children = null;
            }
        }


        /**
         * Clear the cache and re-request the file object
         */
        private void refresh()
        {
            try
            {
                // this.file = ((AbstractFileSystem)
this.file.getFileSystem()).resolveFile(this.file.getName(), false);

                // close the file - this will detach and reattach its resources
(for this thread) on the
                // next access
                this.file.close();
            }
            catch (FileSystemException fse)
            {
                log.error(fse.getLocalizedMessage(), fse);
            }
        }


        /**
         * Recursively fires create events for all children if recursive descent is
         * enabled. Otherwise the create event is only fired for the initial
         * FileObject.
         */
        private void fireAllCreate(FileObject child)
        {
            // Add listener so that it can be triggered
            if (this.fm.getFileListener() != null)
            {
                child.getFileSystem().addListener(child, this.fm.getFileListener());
            }

            try
            {
              // If the child is not closed a file that is present, then deleted
from
              // the file system, then added back would not fire a created event.
              child.close();
            }
            catch (FileSystemException fse)
            {
                log.error(fse.getLocalizedMessage(), fse);
            }
            
            ((AbstractFileSystem) child.getFileSystem()).fireFileCreated(child);

            // Remove it because a listener is added in the queueAddFile
            if (this.fm.getFileListener() != null)
            {
                child.getFileSystem().removeListener(child,
                    this.fm.getFileListener());
            }

            this.fm.queueAddFile(child); // Add

            try
            {

                if (this.fm.isRecursive())
                {
                    if (child.getType() == FileType.FOLDER)
                    {
                        FileObject[] newChildren = child.getChildren();
                        for (int i = 0; i < newChildren.length; i++)
                        {
                            fireAllCreate(newChildren[i]);
                        }
                    }
                }

            }
            catch (FileSystemException fse)
            {
                log.error(fse.getLocalizedMessage(), fse);
            }
        }

        /**
         * Only checks for new children. If children are removed, they'll
         * eventually be checked.
         */
        private void checkForNewChildren()
        {           
            try
            {
                if (this.file.getType() == FileType.FOLDER)
                {
                    FileObject[] newChildren = this.file.getChildren();
                    if (this.children != null)
                    {
                        // See which new children are not listed in the current
children map.
                        Map newChildrenMap = new HashMap();
                        Stack missingChildren = new Stack();

                        for (int i = 0; i < newChildren.length; i++)
                        {
                            newChildrenMap.put(newChildren[i].getName(), new
                                Object()); // null ?
                            // If the child's not there
                            if
                            (!this.children.containsKey(newChildren[i].getName()))
                            {
                                missingChildren.push(newChildren[i]);
                            }
                        }

                        this.children = newChildrenMap;

                        // If there were missing children
                        if (!missingChildren.empty())
                        {

                            while (!missingChildren.empty())
                            {
                                FileObject child = (FileObject)
                                    missingChildren.pop();
                                this.fireAllCreate(child);
                            }
                        }

                    }
                    else
                    {   // TODO: Check - I don't think this block is ever called.
                        // First set of children - Break out the cigars
                        if (newChildren.length > 0)
                        {
                            this.children = new HashMap();
                        }
                        for (int i = 0; i < newChildren.length; i++)
                        {
                            this.children.put(newChildren[i].getName(), new
                                Object()); // null?
                            this.fireAllCreate(newChildren[i]);
                        }
                    }
                }
            }
            catch (FileSystemException fse)
            {
                log.error(fse.getLocalizedMessage(), fse);
            }
        }

        private void check()
        {
            this.refresh();

            try
            {
                // If the file existed and now doesn't
                if (this.exists && !this.file.exists())
                {
                    this.exists = this.file.exists();
                    this.timestamp = -1;

                    // Fire delete event

                    ((AbstractFileSystem)
                        this.file.getFileSystem()).fireFileDeleted(this.file);

                    // Remove listener in case file is re-created. Don't want to
fire twice.
                    if (this.fm.getFileListener() != null)
                    {
                        this.file.getFileSystem().removeListener(this.file,
                            this.fm.getFileListener());
                    }

                    // Remove from map
                    this.fm.queueRemoveFile(this.file);
                }
                else if (this.exists && this.file.exists())
                {

                    // Check the timestamp to see if it has been modified
                    if (this.timestamp !=
                            this.file.getContent().getLastModifiedTime())
                    {
                        this.timestamp =
                            this.file.getContent().getLastModifiedTime();
                        // Fire change event

                        // Don't fire if it's a folder because new file children
                        // and deleted files in a folder have their own event
triggered.
                        if (this.file.getType() != FileType.FOLDER)
                        {
                            ((AbstractFileSystem)
                               
this.file.getFileSystem()).fireFileChanged(this.file);
                        }
                    }

                }

                this.checkForNewChildren();

            }
            catch (FileSystemException fse)
            {
                log.error(fse.getLocalizedMessage(), fse);
            }
        }

    }

}

-- 
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message