Java 7 walkFileTree calling visitFile on directories

java files walk exclude directory
java flatten directory
java directory tree
java monitor a directory
java files.walk depth
java crawl directories
filevisitoption
java directory scanner

Say I have the following directory structure

/root/dir
/root/dir/file1.txt
/root/dir/subdir
/root/dir/subdir/file2.txt

And let's say I'll be using the following visitor:

class MyFileVisitor extends SimpleFileVisitor<Path> {

  @Override
  public FileVisitResult visitFile(final Path file, final BasicFileAttributes attrs)
  throws IOException {
    if(Files.isDirectory(file)) {
      throw new IllegalStateException("WAT!? Visiting directory: "+file.toAbsolutePath().toString());
    }
    System.out.println("Visiting file: "+file.toAbsolutePath().toString());
    return super.visitFile(file, attrs);
  }

If we use the simple overload of walkFileTree:

Files.walkFileTree(Paths.get("/root/dir"), new MyFileVisitor());  

Everything goes according to plan and we see the following output:

Visiting file: /root/dir/file1.txt
Visiting file: /root/dir/subdir/file2.txt

But when we try setting the max depth, things start to break down:

Files.walkFileTree(Paths.get("/root/dir"), EnumSet.noneOf(FileVisitOption.class), 1, new MyFileVisitor());

Output:

Visiting file: /root/dir/file1.txt

java.lang.IllegalStateException: WAT!? Visting directory: /root/dir/subdir

I'm pretty sure this is a bug, but I wanted to ask the community first: Is there something I'm missing and this is actually expected behavior? Thanks for confirming!

Details:

java -version
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

I played around a little bit with your code and added a couple of lines:

/*
 * (non-Javadoc)
 * 
 * @see java.nio.file.SimpleFileVisitor#preVisitDirectory(java.lang.Object,
 * java.nio.file.attribute.BasicFileAttributes)
 */
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
        throws IOException {
    System.out.println(Thread.currentThread().getStackTrace()[1] + " "
            + dir);
    return super.preVisitDirectory(dir, attrs);
}

/*
 * (non-Javadoc)
 * 
 * @see java.nio.file.SimpleFileVisitor#postVisitDirectory(java.lang.Object,
 * java.io.IOException)
 */
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc)
        throws IOException {
    System.out.println(Thread.currentThread().getStackTrace()[1] + " "
            + dir);
    return super.postVisitDirectory(dir, exc);
}

/*
 * (non-Javadoc)
 * 
 * @see java.nio.file.SimpleFileVisitor#visitFileFailed(java.lang.Object,
 * java.io.IOException)
 */
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc)
        throws IOException {
    System.out.println(Thread.currentThread().getStackTrace()[1] + " "
            + file);
    return super.visitFileFailed(file, exc);
}

just no-brainers to see what's going on.

What you call a bug happens here java.nio.file.FileTreeWalker:134:

    // at maximum depth or file is not a directory
    if (depth >= maxDepth || !attrs.isDirectory()) {
        return visitor.visitFile(file, attrs);
    }
Finally, what I think about it:

1.: subdir is part of depth 1 2.: Without the visitFile for deepest directories, you wouldn't even recognize they are there.

So I think that's regular behaviour.

By the way, you could use the attrs in your override of:

@Override
public FileVisitResult visitFile(final Path file,
        final BasicFileAttributes attrs) throws IOException {
    if (attrs.isDirectory()) {
        throw new IllegalStateException("WAT!? Visiting directory: "
                + file.toAbsolutePath().toString());
    }
    System.out
            .println("Visiting file: " + file.toAbsolutePath().toString());
    return super.visitFile(file, attrs);
}

Walking the File Tree (The Java™ Tutorials > Essential Classes , from Java 7 NIO to walk directory tree and calculate sizes of its directories. The method that does all the hard work of traversing directory tree and calling our FileVisitor; BasicFileAttributes public FileVisitResult visitFile(. This is stated in the javadoc of walkFileTree. The visitFile method is invoked for all files, including directories, encountered at maxDepth, unless the basic file attributes cannot be read, in which case the visitFileFailed method is invoked. Refer to the Stack Overflow Question – walkFileTree calling visitFile on directories

I honestly believe java.nio.file.Files#walkFileTree(java.nio.file.Path, java.util.Set, int, java.nio.file.FileVisitor) breaks the contract of java.nio.file.FileVisitor interface.

In a nutshell: visitFile method is invoked on directories too IF they are exactly on the depth level that was provided.

Which really doesn't make sense if you read the documentation of the interface java.nio.file.FileVisitor#visitFile which clearly states:

Invoked for a file in a directory.


Example:

Given the following filesystem structure:

───start_dir
   │   file1.txt
   │
   ├───dir1
   │       file11.txt
   │
   └───dir2
       └───dir21
               file211.txt

Calling walkFileTree works as expected with large enough depth

Files.walkFileTree(Paths.get("start_dir"), EnumSet.noneOf(FileVisitOption.class),1000, new MyFileVisitor());

produces the following invocations which are in line with the interface:

Previsit  dir: start_dir
Previsit  dir: start_dir\dir1
Visit    file: start_dir\dir1\file11.txt
Postvisit dir: start_dir\dir1
Previsit  dir: start_dir\dir2
Previsit  dir: start_dir\dir2\dir21
Visit    file: start_dir\dir2\dir21\file211.txt
Postvisit dir: start_dir\dir2\dir21
Postvisit dir: start_dir\dir2
Visit    file: start_dir\file1.txt
Postvisit dir: start_dir

Calling walkFileTree with depth 1

Files.walkFileTree(Paths.get("start_dir"), EnumSet.noneOf(FileVisitOption.class),1, new MyFileVisitor());

produces the following invocations in which visitFile is invoked on first level directories too in addition to the files:

Previsit  dir: start_dir
Visit    file: start_dir\dir1
Visit    file: start_dir\dir2
Visit    file: start_dir\file1.txt
Postvisit dir: start_dir

Calling walkFileTree with depth 2

Files.walkFileTree(Paths.get("start_dir"), EnumSet.noneOf(FileVisitOption.class),2, new MyFileVisitor());

produces the following invocations in which visitFile is not invoked on first level directories but invoked on second level directories:

Previsit  dir: start_dir
Previsit  dir: start_dir\dir1
Visit    file: start_dir\dir1\file11.txt
Postvisit dir: start_dir\dir1
Previsit  dir: start_dir\dir2
Visit    file: start_dir\dir2\dir21
Postvisit dir: start_dir\dir2
Visit    file: start_dir\file1.txt
Postvisit dir: start_dir

Java 7 Walk File Tree, walkFileTree() to walk through file tree. Here we'll create a file visitor call FindTextFilesVisitor which extend the java.nio.file.SimpleFileVisitor . To get all the text files (files end with .txt ) we override the visitFile() defined by the SimpleFileVisitor . In this method How do I create and delete a file in JDK 7? A visitor of files. An implementation of this interface is provided to the Files.walkFileTree methods to visit each file in a file tree. Usage Examples: Suppose we want to delete a file tree. In that case, each directory should be deleted after the entries in the directory are deleted. Path start =

This behavior is specified in the documentation of Files::walkFileTree, so it is intentional and not a bug:

The visitFile method is invoked for all files, including directories, encountered at maxDepth

In defense of visitFile being invoked on directories, directories are implicitly called files throughout the documentation of the java.nio.file classes. For example, the documentation of Files::walkFileTree itself says, "This method walks a file tree rooted at a given starting file," and that "starting file" may (of course) be a directory.

Also note that visitFileFailed may also be invoked on directories, as explained in the documentation:

Where the file is a directory, and the directory could not be opened, then the visitFileFailed method is invoked with the I/O exception

Franz Ebner showed the code where this behavior occurs and I agree with why he thinks it is intentional: "Without the visitFile for deepest directories, you wouldn't even recognize they are there." I also concur with checking the file type using the attrs parameter if you want only regular files (or symbolic links).

How to recursively list all text files in a directory?, A quick and practical guide to Java NIO2 FileVisitor. TERMINATE – stops the file tree traversal and no further directories or files are We just have to call the static walkFileTree API of the Files class and pass to it an 7. 8. 9. @Override. public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {. However the JavaDoc of walkFileTree explains why this results in the desired behaviour: For each file encountered this method attempts to read its java.nio.file.attribute.BasicFileAttributes. If the file is not a directory then the visitFile method is invoked with the file attributes.

A Guide To NIO2 FileVisitor, The good thing is that Java 7 introduced a very convenient way of doing it using FileVisitor. When you want to recursively visit specific directory, you call one of the Path walkFileTree(Path start, FileVisitor<? super Path> visitor); of a directory were visited including its subdirectories); visitFile – Called  Java SE 7 has introduced many strong features for file handling. In Java NIO 2, we have the API that can traverse or walk the file tree. The class java.nio.file.Files has a method that is walkFileTree and an interface named java.nio.file. FileVisitor plays the role to traverse the directory structure.

Recursively walking directory using Java NIO, Walking a directory tree is a new feature in Java 7 and you will need to know a number of interfaces walkFileTree(Path startingDir, FileVisitor<? super Path> visitor); FileVisitResult visitFile(T file, BasicFileAttributes attrs); FileVisitResult CONTINUE; } } } #A Starting directory #1 Calls to walkTheTree #2  By default, walkFileTree does not follow symbolic links. The visitFile method is invoked for files. If you have specified the FOLLOW_LINKS option and your file tree has a circular link to a parent directory, the looping directory is reported in the visitFileFailed method with the FileSystemLoopException.

Dealing with Directories and Directory Trees in Java 7.0, The Java 7 NIO library allows you to walk a file tree and visit each file in the tree. You do this by implementing a FileVisitor and then calling Files.walkFileTree visitFile : Invoked for a file in a directory. walkFileTree(dir, new SimpleFileVisitor<Path>() { @Override public FileVisitResult visitFile(Path file,  The following are Jave code examples for showing how to use walkFileTree() of the java.nio.file.Files class. You can vote up the examples you like. Your votes will be used in our system to get more good examples.

Comments
  • Yah, you make a good point; I guess the method name didn't make sense to me. Perhaps visitPath would be better if it intends to visit both files and directories as it does now. Thanks for the clarification.
  • If you keep the depth unlimited it shows only regular files. But in this case every directory is hooked 2 times as well.
  • I felt the same way at the time that I posted the question. I think I dug into the java code a bit more and convinced myself that the unexpected behavior was necessary for some reason; but I can't recall why anymore. Regardless; the method name and javadoc could definitely use some improvement if it will continue visiting directories at maxDepth.