Read csv into a list in clojure

Related searches

I know there are a lot of related questions, I have read them but still have not gained some fundamental understanding of how to read-process-write. Take the following function for example which uses clojure-csv library to parse a line

(defn take-csv
  "Takes file name and reads data."
  [fname]
  (with-open [file (reader fname)]
    (doseq [line (line-seq file)]
      (let [record (parse-csv line)]))))

What I would like to obtain is data read into some collection as a result of (def data (take-csv "file.csv")) and later to process it. So basically my question is how do I return record or rather a list of records.

"doseq" is often used for operations with side effect. In your case to create collection of records you can use "map":

(defn take-csv
  "Takes file name and reads data."
  [fname]
  (with-open [file (reader fname)]
    (doall (map (comp first csv/parse-csv) (line-seq file)))))

Better parse the whole file at ones to reduce code:

(defn take-csv
  "Takes file name and reads data."
  [fname]
  (with-open [file (reader fname)]
    (csv/parse-csv (slurp file))))

You also can use clojure.data.csv instead of clojure-csv.core. Only should rename parse-csv to take-csv in previous function.

(defn put-csv [fname table]
  (with-open [file (writer fname)]
    (csv/write-csv file table)))

clojure/data.csv, If you want to be careful not to read the csv file into memory the first approach is preferable. Parsing into maps. Data.csv parses lines of a csv file into a vector of� read-csv function Usage: (read-csv input & options) Reads CSV-data from input (String or java.io.Reader) into a lazy sequence of vectors. Valid options are :separator (default \,) :quote (default \") Source

With all the things you can do with .csv files, I suggest using clojure-csv or clojure.data.csv. I mostly use clojure-csv to read in a .csv file.

Here are some code snippets from a utility library I use with most of my Clojure programs.

from util.core

    (ns util.core
      ^{:author "Charles M. Norton",
        :doc "util is a Clojure utilities directory"}

      (:require [clojure.string :as cstr])
      (:import java.util.Date)
      (:import java.io.File)
      (:use clojure-csv.core))

(defn open-file
"Attempts to open a file and complains if the file is not present."

[file-name]
(let [file-data (try 
               (slurp file-name)
               (catch Exception e (println (.getMessage e))))]
  file-data))

(defn ret-csv-data
"Returns a lazy sequence generated by parse-csv.
 Uses open-file which will return a nil, if
 there is an exception in opening fnam.

 parse-csv called on non-nil file, and that
 data is returned."

[fnam]
(let [csv-file (open-file fnam)
      inter-csv-data (if-not (nil? csv-file)
                       (parse-csv csv-file)
                        nil)

      csv-data 
        (vec (filter #(and pos? (count %) 
           (not (nil? (rest %)))) inter-csv-data))]

    (if-not (empty? csv-data)
      (pop csv-data)
       nil)))

(defn fetch-csv-data
    "This function accepts a csv file name, and returns parsed csv data,
     or returns nil if file is not present."

    [csv-file]
        (let [csv-data (ret-csv-data csv-file)]
            csv-data))

Once you've read in a .csv file, then what you do with its contents is another matter. Usually, I am taking .csv "reports" from one financial system, like property assessments, and formatting the data to be uploaded into a database of another financial system, like billing.

I will often either zipmap each .csv row so I can extract data by column name (having read in the column names), or even make a sequence of zipmap'ped .csv rows.

clojure.data.csv, Usage: (read-csv input & options) Reads CSV-data from input (String or java.io. Reader) into a lazy sequence of vectors. Valid options are :separator (default \� Scan the the directory tree to create a list of CSV files we want to scan. Autodetect the data type of each column across all files in each directory, and store the schema just outside the directory, so that it can be modified as needed. Create the table from the autodetected schema. Load the CSV files into SQL tables.

Just to add this good answers, here is a full example

First, add clojure-csv into your dependencies

(ns scripts.csvreader
  (:require [clojure-csv.core :as csv]
            [clojure.java.io :as io]))

(defn take-csv
 "Takes file name and reads data."
 [fname]
 (with-open [file (io/reader fname)]
  (-> file
      (slurp)
      (csv/parse-csv))))

usage (take-csv "/path/youfile.csv")

with-open - clojure.core, (defn write-csv-file "Writes a csv file using a key and an s-o-s (sequence of Try to read 3 lines of text from a test file and return them (to the REPL) ;; as a list:� CSV raw data is not utilizable in order to use that in our Python program it can be more beneficial if we could read and separate commas and store them in a data structure. We can convert data into lists or dictionaries or a combination of both either by using functions csv.reader and csv.dictreader or manually directly

zipmap - clojure.core, transform a CSV file to an array of maps using the header line as keys user=> ( defn csv-map "ZipMaps header as keys and values from lines." [head & lines]� Now let’s see how to import the contents of this csv file into a list. Read a CSV into list of lists in python. There are different ways to load csv contents to a list of lists, Import csv to a list of lists using csv.reader. Python has a built-in csv module, which provides a reader class to read the contents of a csv file. Let’s use that,

Read CSV Data File with Clojure – Yet Another Blog in Statistical , Import CSV as Dictionary List � An Example of Merge Layer in Keras � Modeling Practices of Loss Forecasting for Read CSV Data File with Clojure READ CSV FILE. user=> ( def ds1 (read-dataset "../data/credit_count.csv" :header true : delim \,)) Two Ways to Select Rows in IncanterIn "clojure". Which approach to choose depends on the application. If the csv file isn't huge the second approach will often work well. If you want to be careful not to read the csv file into memory the first approach is preferable. Parsing into maps. Data.csv parses lines of a csv file into a vector of strings.

Save this file as “crickInfo.csv”. How Python Read CSV File into Array List? As like any text file you can read and split the content using comma operator. But there is an automated module called CSV. Just import it and it will do the things for you. In this post, I will show you how to read a CSV file in python?

Comments
  • Nice. Now all is left is to put-csv, will you help?
  • Pay attention to doall : doall can be used to force any effects. Walks through the successive nexts of the seq, retains the head and returns it, thus causing the entire seq to reside in memory at one time.
  • Sure. But when you try realize next elements of lazy seq result you would get an exception because the file would be already closed at that moment. In order to fix that and reduce memory consumption you could pass a function which does the work with the result of a map to the take-csv as a parameter and use it within a with-open scope.
  • Thank you. Still it would be interesting to hear your comment to reading the whole file into memory, which might be a problem for very large files.
  • As far as I can tell it's a lazy sequence of maps. clojure-csv returns a lazy sequence, and zipmap does too. That's how I do it.