pandas map makes values NaN

pandas dataframe map
pandas map nan to 0
pandas map returns nan
pandas map example
pandas index map
pandas map values from one column to another
pandas map vs apply
map function to pandas column

I'm trying to map my values that i want to change. When i apply 'map' like this >> df[column].map(dictionary), the values that are not in the dictionary convert to NaN. I think the reason is that there are no matched values in the series, right? If so, nothing should be applied instead converting to NaN? How can i solve this problem using df.map() instead of df.replace()?

df1 = pd.Series(['a','b','c','d'])
df
0    a
1    b
2    c
3    d
dtype: object

mapping = {'a' : 0, 'b' : 1, 'c' : 2}
df1.map(mapping)
0    0.0
1    1.0
2    2.0
3    NaN
dtype: float64

or

df1 = pd.Series(['a','b','c','d'])
df
0    a
1    b
2    c
3    d
dtype: object

mapping = {'k' : 0, 'e' : 1, 'f' : 2}
df1.map(mapping)

0   NaN
1   NaN
2   NaN
3   NaN
dtype: float64

This behavior is intended. Since mapping can not be applied the value is NaN. In order to use mapping you have to create a specific value that does not change your data (if you do multiplication that would be 1, if you do addition then 0) and add that value to your mapping.

Alternatively you could replace all NaN values after you have done the mapping with a neutral value like 0.0.

Either way is much more work then to simply use replace.

pandas.Series.map, argfunction, collections.abc.Mapping subclass or Series. Mapping correspondence. na_action{None, 'ignore'}, default None. If 'ignore', propagate NaN values,  The official documentation for pandas defines what most developers would know as null values as missing or missing data in pandas. Within pandas, a missing value is denoted by NaN . In most cases, the terms missing and null are interchangeable, but to abide by the standards of pandas, we’ll continue using missing throughout this tutorial.

If you insist on map pass a callable instead

df.map(lambda x: mapping.get(x,x))

pandas.Series.map, When arg is a dictionary, values in Series that are not in the dictionary (as keys) are converted to NaN . However, if the dictionary is a dict subclass that defines  Depending on the scenario, you may use either of the 4 methods below in order to replace NaN values with zeros in pandas DataFrame: (1) For a single column using pandas: (2) For a single column using numpy: (3) For an entire DataFrame using pandas: (4) For an entire DataFrame using numpy: Let’s now review how to apply each of the 4 methods

To change the default value, you could add a function (func, here):

mapping = {'k' : 0, 'e' : 1, 'f' : 2}
mapping.setdefault('Default', 'write watherver you want here')
def func(x, mapping):
    try:
        tmp=mapping[x]
        return(tmp)
    except:
        return('default value')
df1.map(lambda x: func(x, mapping))

pandas.Series.map, If 'ignore', propagate NaN values, without passing them to the mapping correspondence. Returns: Series. Same index as caller. See also. Series. Values considered “missing”¶ As data comes in many shapes and forms, pandas aims to be flexible with regard to handling missing data. While NaN is the default missing value marker for reasons of computational speed and convenience, we need to be able to easily detect this value with data of different types: floating point, integer, boolean, and general object.

pandas.DataFrame.fillna, Fill NA/NaN values using the specified method. Parameters. valuescalar, dict, Series, or DataFrame. Value to use to fill holes (e.g. 0),  pandas map makes values NaN. Ask Question Asked 1 year, 8 months ago. Active 1 year, 8 months ago. Viewed 1k times 3. I'm trying to map my values that i want to

pd.Series.map never maps NAs through a dictionary · Issue #17648 , Code Sample import pandas as pd, numpy as np print(pd.Series([1, 2, np.nan]).​map({1: "a", 2: "b", np.nan: "c"})) This prints: 0 a 1 b 2 NaN dtype: a descriptive label by the codebook, analogously to the non-missing values. Detect existing (non-missing) values. Return a boolean same-sized object indicating if the values are not NA. Non-missing values get mapped to True. Characters such as empty strings '' or numpy.inf are not considered NA values (unless you set pandas.options.mode.use_inf_as_na = True ). NA values, such as None or numpy.NaN, get mapped to False

Series.map should return default dictionary values rather than NaN , collections.Counter and collections.defaultdict both have default values. However​, pandas.Series.map does not respect these defaults and  This prints: 0 a 1 b 2 NaN dtype: object Problem description. The parameter na_action of pd.Series.map is ignored if the first argument of map is a dictionary. . Rather than mapping NAs if na_action is None and passing them through unaltered if it's "ignore", pandas always passes NAs through unaltered, as if na_action were "ig

Comments
  • "How can i solve this problem using df.map() instead of df.replace()?" <- I don't understand. It sounds like you already know that replace has the behaviour you want. Why do you think you need to use map?
  • Actually, it just for my coding skill. I want to know that map can handle those problem too.