Parsing datetime strings containing nanoseconds

numpy datetime64
python nanoseconds
python datetime milliseconds
python convert nanoseconds since epoch to datetime
datetime strptime microseconds
python datetime format for milliseconds
convert object to datetime python
python datetime iso format milliseconds

I have some log files with times in the format HH:MM::SS.nano_seconds (e.g. 01:02:03.123456789). I would like to create a datetime in python so I can neatly do math on the time (e.g. take time differences). strptime works well for microseconds using %f. Do the Python datetime and time modules really not support nanoseconds?

You can see from the source that datetime objects don't support anything more fine than microseconds. As pointed out by Mike Pennington in the comments, this is likely because computer hardware clocks aren't nearly that precise. Wikipedia says that HPET has frequency "at least 10 MHz," which means one tick per 100 nanoseconds.

If you can live with throwing out the last three digits (which probably aren't too meaningful anyway), you could parse this by just slicing the input string to have only six digits after the decimal point and parsing with %f. Otherwise, it looks like you'll have to implement the subtraction yourself.


Much later update: numpy and pandas now each have (somewhat different) support for timestamps that includes the possibility of tracking nanoseconds, which are often good solutions. See the other answers for how.

Python 3.7+ also has time.time_ns and related functions in time (PEP 564), but still no support for nanoseconds in datetime.

Will DateTime.Parse be able to deal with nanoseconds? · Issue , Issue Title DateTime can not deal with the fraction of second more than Parse , users do not care about how many digits of the string, and the  Parsing datetime strings containing nanoseconds (3 answers) Closed 7 years ago . I have a timestamp in epoch time with nanoseconds - e.g. 1360287003083988472 nanoseconds since 1970-01-01.

You can quite naturally use nanoseconds and even more precise time units (ps, fs, as) with numpy. Numpy has its own Datetimes and Timedeltas implementation, so you can try np.datetime64:

import numpy as np
def str_to_ns(time_str):
     """
     input: time in a format `hh:mm:ss.up_to_9_digits`
     """
     h, m, s = time_str.split(":")
     int_s, ns = s.split(".")
     ns = map(lambda t, unit: np.timedelta64(t, unit),
              [h,m,int_s,ns.ljust(9, '0')],['h','m','s','ns'])
     return sum(ns)

Then you can use this function in a following way:

>>> src = "1:2:34.123456789"
>>> out = str_to_ns(src)
>>> print(out)
3754123456789 nanoseconds
>>> out / np.timedelta64(1,'h')
1.0428120713302778
>>> out / np.timedelta64(1,'m')
62.568724279816664
>>> out / np.timedelta64(1,'s')
3754.123456789

Arithmetic is also possible:

>>> t1, t2 = str_to_ns("1:0:12.12345678"), str_to_ns("1:0:12.12")
>>> t1 - t2
numpy.timedelta64(3456780,'ns')

I agree that it's not that natural, but in this manner you can achieve arbitrary high time precision with just numpy.

DateTime.ParseExact Method (System), public static DateTime ParseExact (string s, string format, IFormatProvider provider); A string that contains a date and time to convert. dateString); } // Parse a date that includes seconds and milliseconds // by using the French (​France) and  Possible duplicate of Parsing datetime strings containing nanoseconds – Adrian Shum Jul 18 '19 at 2:01 OK, based on the answers below, this is required json['time'].replace('000', '', 1) to remove the additional 3 zeros in the microseconds. – cube Jul 18 '19 at 3:14

This is an old thread, but still...

You can use Pandas functionality to achieve this. I had timestamps like '2019-03-22T14:00:01.700311864Z' which I converted to a Timestamp by:

    firstStamp = pd.to_datetime(firstStampString, format='%Y-%m-%dT%H:%M:%S.%fZ')
    lastStamp = pd.to_datetime(lastStampString, format='%Y-%m-%dT%H:%M:%S.%fZ')

    deltaTime = lastStamp - firstStamp

This works fine.

pandas.to_datetime, Warning: dayfirst=True is not strict, but will prefer to parse with day first (this is a eg “%d/%m/%Y”, note that “%f” will parse all the way up to nanoseconds. datetime strings, and if it can be inferred, switch to a faster method of parsing them. Everything related to datetimes is handled through the rust-chrono package, which is precise to the nanosecond and can parse ISO 8601 formatted strings, truncating digits more granular than nanoseconds. extern crate chrono; use chrono:: *; fn main {let s = "2016-06-10T21:42:24.760738998Z"; let dt = s.parse:: < DateTime < UTC >> ().unwrap (); println!

How can I convert a string with fractional seconds into a datetime , How can I convert a string with fractional Learn more about datetime;, format;, fractional;, seconds;, milliseconds;, nat;, parse;, format, identifiers; MATLAB. Parse date and time strings in .NET. 02/15/2018; 7 minutes to read +11; In this article. Parsing strings to convert them to DateTime objects requires you to specify information about how the dates and times are represented as text.

chrono::DateTime - Rust, note: this is not the number of nanoseconds since January 1, 1970 0:00:00 UTC Parses a string with the specified format string and returns a new DateTime  strftime to parse time, eg “%d/%m/%Y”, note that “%f” will parse all the way up to nanoseconds. unit string, default None. unit of the arg (D,s,ms,us,ns) denote the unit, which is an integer or float number. This will be based off the origin. Example, with unit=’ms’ and origin=’unix’ (the default), this would calculate the number of milliseconds to the unix epoch start. infer_datetime_format boolean, default False

Issue 15443: datetime module has no support for nanoseconds , Bob has a web2py website that has some data with nanosecond granularity. Support for nanoseconds in datetime would make it possible to properly parse these Are the nanosecond timestamps timestamps or strings? Dismiss Join GitHub today. GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.

Comments
  • possible duplicate of Get POSIX/Unix time in seconds and nanoseconds in Python?
  • @MikePennington That question deals with getting a clock time with nanoseconds, not parsing them and doing math on them independent of an actual clock.
  • @Dougal, the question is very relevant since they point out that nanosecond time precision requires platform support, and most do not. In that question, string formatting of nanoseconds is discussed as well
  • @MikePennington Well, it doesn't solve the question or give a reason why it's hard to solve -- just shows why standard datetime approaches don't do it. This problem only needs to deal with nanoseconds in the abstract, not do anything involving actual system times. It's a useful link but not a duplicate.
  • Maybe a solution with PEP 564 - Add new time functions with nanosecond resolution (October 2017) python.org/dev/peps/pep-0564
  • The system clock is not the only source of time one might want to use datetime and timedelta with. It is annoying (bordering on myopic idiocy) that they aren't implemented using nsecs instead of usecs.
  • @travc: There is open CPython issue: datetime module has no support for nanoseconds
  • Here in 2016, this is still a problem. I agree with @travc that it is idiocy to not support nanoseconds, since many scientists require such precision when working with Python
  • @sirgogo, just a note that numpy's datetime64 dtype supports down to attoseconds (sort of).
  • See also PEP 564 - Add new time functions with nanosecond resolution (October 2017) python.org/dev/peps/pep-0564
  • related: github.com/pandas-dev/pandas/issues/29461