Directory.EnumerateFiles => UnauthorizedAccessException

There is a nice new method in .NET 4.0 for getting files in a directory in a streaming way via enumeration.

The problem here is that if one wishes to enumerate all files one may not know in advance which files or folders are access protected and can throw an UnauthorizedAccessException.

To reproduce, one can just run this fragment:

foreach (var file in Directory.EnumerateFiles(@"c:\", "*", SearchOption.AllDirectories))
{
   // whatever
}

Before this .NET method existed it was possible to achieve roughly the same effect by implementing a recursive iterator on the string-array returning methods. But it's not quite as lazy as the new .NET method is.

So what to do? Can the UnauthorizedAccessException be suppressed or is a fact of life when using this method?

Seems to me that the method should have an overload accepting an action to deal with any exceptions.

Ths issue with the above answer is that is does not take care of exception in sub directories. This would be a better way to handling those exceptions so you get ALL files from ALL subdirectories except those with threw an access exception:

    /// <summary>
    /// A safe way to get all the files in a directory and sub directory without crashing on UnauthorizedException or PathTooLongException
    /// </summary>
    /// <param name="rootPath">Starting directory</param>
    /// <param name="patternMatch">Filename pattern match</param>
    /// <param name="searchOption">Search subdirectories or only top level directory for files</param>
    /// <returns>List of files</returns>
    public static IEnumerable<string> GetDirectoryFiles(string rootPath, string patternMatch, SearchOption searchOption)
    {
        var foundFiles = Enumerable.Empty<string>();

        if (searchOption == SearchOption.AllDirectories)
        {
            try
            {
                IEnumerable<string> subDirs = Directory.EnumerateDirectories(rootPath);
                foreach (string dir in subDirs)
                {
                    foundFiles = foundFiles.Concat(GetDirectoryFiles(dir, patternMatch, searchOption)); // Add files in subdirectories recursively to the list
                }
            }
            catch (UnauthorizedAccessException) { }
            catch (PathTooLongException) {}
        }

        try
        {
            foundFiles = foundFiles.Concat(Directory.EnumerateFiles(rootPath, patternMatch)); // Add files from the current directory
        }
        catch (UnauthorizedAccessException) { }

        return foundFiles;
    }

How to: Enumerate directories and files, Returns an enumerable collection of file information in the current directory. In this article. Definition; Overloads; EnumerateFiles(); EnumerateFiles(String)  Enumerable collections provide better performance than arrays when you work with large collections of directories and files. To enumerate directories and files, use methods that return an enumerable collection of directory or file names, or their DirectoryInfo, FileInfo, or FileSystemInfo objects. If you want to search and return only the names

I Couldn't get the above to work, but here is my implementation, i've tested it on c:\users on a "Win7" box, because if has all these "nasty" dirs:

SafeWalk.EnumerateFiles(@"C:\users", "*.jpg", SearchOption.AllDirectories).Take(10)

Class:

public static class SafeWalk
{
    public static IEnumerable<string> EnumerateFiles(string path, string searchPattern, SearchOption searchOpt)
    {   
        try
        {
            var dirFiles = Enumerable.Empty<string>();
            if(searchOpt == SearchOption.AllDirectories)
            {
                dirFiles = Directory.EnumerateDirectories(path)
                                    .SelectMany(x => EnumerateFiles(x, searchPattern, searchOpt));
            }
            return dirFiles.Concat(Directory.EnumerateFiles(path, searchPattern));
        }
        catch(UnauthorizedAccessException ex)
        {
            return Enumerable.Empty<string>();
        }
    }
}

DirectoryInfo.EnumerateFiles Method (System.IO), I created some helper methods to solve this which I blogged about earlier this year. One version takes a regex pattern \.mp3|\.mp4 , and the  Returns an enumerable collection of file information in the current directory. EnumerateFiles(String) Returns an enumerable collection of file information that matches a search pattern. EnumerateFiles(String, EnumerationOptions) Returns an enumerable collection of file information that matches the specified search pattern and enumeration options.

How to filter Directory.EnumerateFiles with multiple criteria?, Directory EnumerateFiles(String, String, SearchOption) Returns an enumerable collection of file names that match a search pattern in a specified path, and optionally searches subdirectories. This is a very old question (already suitably answered by @MikaelSvenson ), but another option is to use the Enumerable extension .Union(), like so: foreach (var file in Directory.EnumerateFiles(path, ".mp3", SearchOption.AllDirectories).Union(Directory.EnumerateFiles(path, ".wma", SearchOption

I'm late, but I suggest using observable pattern instead:

public class FileUtil
{
  private static void FindFiles_(string path, string pattern,
    SearchOption option, IObserver<string> obs, CancellationToken token)
  {
    try
    {
      foreach (var file in Directory.EnumerateFiles(path, pattern,
        SearchOption.TopDirectoryOnly))
      {
        if (token.IsCancellationRequested) break;
        obs.OnNext(file);
      }

      if (option != SearchOption.AllDirectories) return;

      foreach (var dir in Directory.EnumerateDirectories(path, "*", 
        SearchOption.TopDirectoryOnly))
      {
        FindFiles_(dir, pattern, option, obs, token);
      }
    }
    catch (UnauthorizedAccessException) { }
    catch (PathTooLongException) { }
    catch (IOException) { }
    catch (Exception err) { obs.OnError(err); }
  }

  public static IObservable<string> GetFiles(string root, string pattern,
    SearchOption option)
  {
    return Observable.Create<string>(
      (obs, token) =>
        Task.Factory.StartNew(
          () =>
          {
            FindFiles_(root, pattern, option, obs, token);
            obs.OnCompleted();
          },
          token));
  }
}

C# Tutorial - C# Directory EnumerateFiles(String, String , HI I would like to understand what is the best option in performance and use of memory. Imagine a folder with 10000 files and I want to find the  The problem is the Directory.EnumerateFiles method itself. And I don’t think there is a way to handle the problem. And I don’t think there is a way to handle the problem. You have to resort to SearchOption.TopDirectoryOnly and handle the recursion yourself, I believe.

What is the difference between Directory.EnumerateFiles vs , EnumerateFiles. Many methods on the Directory type are available. We call them as static methods: to call EnumerateFiles, we can use Directory. What is the default read order for the Directory.EnumerateFiles method? Is it consistent? In my experience so far it seems to be by the date the files were created but I haven't been able to find confirmation of this.

C# Directory Type, GetFiles and EnumerateFiles methods from System.IO. Directory.GetFiles. This returns the file names in a folder. It returns a string array—this contains the full  The EnumerateDirectories and GetDirectories methods differ as follows: When you use EnumerateDirectories, you can start enumerating the collection of names before the whole collection is returned; when you use GetDirectories, you must wait for the whole array of names to be returned before you can access the array. Therefore, when you are

C# Directory.GetFiles Example (Get List of Files), The line won't work, because there is no facility AFAIK to specify multiple extensions in either Directory file list method: GetFiles and  Directory.GetFiles is fast when used on small directories with few files. But for large directories Directory.EnumerateFiles is faster.EnumerateFiles, GetFiles Benchmark.

Comments
  • Yes, your Dump() method should be resilient to problems with the files it is trying to dump. Give it an overload.
  • That's not my problem Hans. The problem is that foreach-ing over the file iterator (EnumerateFiles) provokes an UnauthorizedAccessException and that in turn halts further enumeration, which is not desirable when one wants an exhaustive result set.
  • @Hans - The Dump() method is not the problem here, it just walks through the string enumeration. The problem is the Directory.EnumerateFiles method itself. And I don’t think there is a way to handle the problem. You have to resort to SearchOption.TopDirectoryOnly and handle the recursion yourself, I believe.
  • @Mormegil - There's an easy workaround. It kind of sucks though...
  • This (and other reasons) is why I finally ended up writing a wrapper for NtQueryDirectoryFile myself.
  • Nice answer. Thank you!
  • I also ran into a problem with this. The solution I came up with can be found at stackoverflow.com/questions/13130052/…. It behaves as a true enumerable in the sense that it only does work if you ask for the next item from it.
  • what happens if this hits an a restricted file and the exception is thrown/ignored. Wouldn't it just stop there and ignore all files behind this point?
  • Not bad, but it wohn't work for my scenario: I need it to continue and move on when it meets an exception: the only difference between a safe-walk and a normal walk is that the safe-walk just stops enumeration whilst the normal method stops with an exception. I need it to continue and ignore any exceptions in the sense that it should enumerate all directories it can and just skip the ones it doesn't have access to. That, unfortunately it appears, requires a new implementation of the BCL implementation.
  • ...I would have no issue using it in production if it worked ;-) ... but even then it would need a few modifications: for example you don't want to catch all exceptions, it should just be UnauthorizedAccessException or at least it should be filterable via a lambda.
  • Unfortunately, MoveNext() doesn't advance its position when it throws an exception.
  • Something like DirectoryEnumerationPolicy.SkipUnauthorizedPaths would be nice as an additional argument to Directory.Enumerate{Directories,Files,FileSystemEntries}.
  • The fastest option by far. Most solutions load all entries of one level into memory before returning them - which builds up quickly on recursion. This solution magically hopes for MoveNext to work even if it throws. Risky - tested with netstandard 2.0 and it worked to search C:\ recursively while $Recyclebin and system dirs were skipped.