Python - loop through files in different folders

I want to extract data from multiple files nested within subfolders.

e.g. folder structure

A/B/C/D.dat
A/B/E/F.dat
A/B/G/H.dat

The code I came up with is:

import os
values = 2
doc = []
rootdir = 'C:/A/B'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        if file.endswith('.dat'):
            with open (file, 'rt') as myfile:
                    current_line = 0
                    for mylines in myfile:
                            if current_line == values:
                                doc.append()
                                break
                            current_line += 1
            continue

print(doc)

Error I struggle to solve:

...with open (file, 'rt') as myfile:
IOError: [Errno 2] No such file or directory: 'D.dat'

sounds like you are looking for the third line of all the .dat files in the subdirectories. There are using pathlib.Path you can do quite a lot of this in a few simple steps.

from pathlib import Path
doc = []
line_number_of_each_file = values = 2

for file in Path('C:/A/B').rglob('*.dat'):
    doc.append(file.readtext().splitlines()[line_number_of_each_file])

print(doc)

How can I iterate over files in a given directory?, A short block of code to demonstrate how to iterate over files in a directory and do some action with them. import os directory = 'the/directory/you/want/to/use' for filename in os.listdir(directory): if filename.endswith(".txt"): f = open(filename) lines = f.read() print (lines[10]) continue else: continue. In Python, the os.listdir() method lists files and folders in a given directory. The method does not return special entries such as ‘.’ and ‘..’, which the operating system uses to navigate through different directories. It also does not return files and folders beyond the first level of folders.

Though your solution is not the cleanest. The bug you are getting comes from

            with open (file, 'rt') as myfile:

which should be replaced with

            with open (subdir + "/" + file, 'rt') as myfile:

Iterating over files with Python — CARREFAX, How to iterate through the contents of a directory in Python. Iterating through the contents of a directory finds both the files and directories contained within a path � I have a python script that will loop through all folders and sub folders in a specified directory searching for mxd's with a specific data source. If the data source is found it will replace that data source with a new one. The script I have works, but it is very slow. I did it on a test folder with 3 sub folders and a total of 9 maps.

Error is due to missing of complete file path. So you need make sure that "A/B/C/D.dat" should be there in file which you are trying to open as myfile.

you can add the below snippet to your logic to achieve it.

for subdir, dirs, files in os.walk(rootdir): for file in files: filepath=subdir+'/'+file

How to iterate through the contents of a directory in Python, Otherwise, an iterative macro could work to loop through different structures. Whenever I hear loop, that indicates iterative. Let me know if that� ''' For the given path, get the List of all files in the directory tree ''' def getListOfFiles(dirName): # create a list of file and sub directories # names in the given directory listOfFile = os.listdir(dirName) allFiles = list() # Iterate over all the entries for entry in listOfFile: # Create full path fullPath = os.path.join(dirName, entry

I had a similar problem. My file structure is something like this:

project
|__dir1
|  |__file_to_read.txt
|
|__dir2
   |__file_reader.py

In order to actually find the other file, I have to go out one directory, to the parent dir of my .py file. I used this code originally:

import os

current_path = os.path.dirname(__file__)

file_to_read = os.path.relpath('project/dir1/file_to_read', current_path)

This worked for me, but I later changed over to a different version. The reason is not for any reason you'll have to worry about, other than evidently this next module is better for path crawling than os.

from pathlib import Path

parent = Path.cwd().parent
file_to_read = Path(f'{parent}/project/dir1/file_to_read.txt').resolve()

Maybe this would be more preferable, as it's more highly recommended to me. I hope this helps your problem.

Loop through folders in directory, The Python os.listdir() method returns a list of every file and folder in a and '..', which the operating system uses to navigate through different directories. Finally, we create a for loop that iterates through every item in the list� To get a list of all the files and folders in a particular directory in the filesystem, use os.listdir() in legacy versions of Python or os.scandir() in Python 3.x. os.scandir() is the preferred method to use if you also want to get file and directory properties such as file size and modification date.

The issue here is that you are trying to call filename.dat. Instead you should access A/B/C/filename.dat . Join the pathname + filename for this:

import os
values = 2
doc = []
rootdir = 'A/B/C/'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        if file.endswith('.txt'):
            print(file)
            file = os.path.join(rootdir,subdir,file)
            with open (file, 'rt') as myfile:
                    current_line = 0
                    for mylines in myfile:
                            if current_line == values:
                                doc.append(mylines)
                                break
                            current_line += 1
            continue

print(doc)

Python List Files in a Directory: Step-By-Step Guide, How does one loop through the directory names of the current path in python? How many types of loops are there in Python and how we implement it? 39 Responses to “Python: iterate (and read) all files in a directory (folder)” Dt Says: December 23rd, 2008 at 11:38. works just fine for me, only important change to the code that i had to make was turning print into a function because im using python 3.0, i also set it to read files with *all* extensions.

How does one loop through the directory names of the current path , Just a thought. Line 5 calls out the directory but line 9 does not. through the loop pet equals ‘cat’, and so on. We loop through the indented block of code for each item in the pets list. After we process the last item, ‘Maine Coon cat’ we exit the loop. lines 4 & 5 : We set the name length to the length of whatever string is named ‘pet’. For example, the first time through the loop

Python loop over files in directory then [SOLVED], os.listdir(my_path) will get you everything that's in the my_path directory - files and directories. You can use it as follows:>>> import os� In the above structure, there is an __init__.py file added to all the folders. This file ensures that the folder is included as a python package. This file is typically empty, however, we could also add the generic module import statement in this file. Import files: Import the files using the syntax from <folder>.<filename> import <module

How can I iterate over files in a given directory in Python?, See my attached demo where it recurses through a folder and all subfolders. @ Brando Miranda: "list comprehensions" exist only in Python. recommends and shows in all examples that require reading multiple files from a folder, e.g.:. What you may know as a file is slightly different in Python. In Windows, for example, a file can be any item manipulated, edited or created by the user/OS. That means files can be images, text documents, executables, and excel file and much more. Most files are organized by keeping them in individual folders.