Change dataframes in dict

python pandas convert dataframe to dictionary with multiple values
pandas dataframe from dict of dicts
dataframe to dictionary with one column as key
nested dictionary to dataframe
convert dataframe to dictionary with one column as key
pandas dataframe from list of dicts
pandas dataframe to dict two columns
pandas unpack dictionary

Please help me to understand how to change dataframes in dictionary.

Let's consider the simplest case and create two dataframes and construct the dict from them.

dates = pd.date_range('20130101',periods=6)
df1 =pd.DataFrame(np.random.randn(6,4),index=dates,columns=list('ABCD'))
df2 =pd.DataFrame(np.random.randn(6,4),index=dates,columns=list('ABCD'))
DICTOR={}
DICTOR['d1']=df1
DICTOR['d2']=df2
m=DICTOR

Now I want to exclude rows from DataFrames inside dict m, for example rows with where values in B columns are zero or negative.

I tried following code:

for name,df in m.items():
     for index, row in df.iterrows():
         if df.at[index,'B']<0:
             df.drop(index,axis=0)

or:

for name,df in m.items():
    df=df[df.B>0]

but it does not work.

I guess my problem is due to mutable/immutable objects, but i'm not sure.

Change your loop to this:

for name,df in m.items():
     for index, row in df.iterrows():
         if df.at[index,'B']<0:
             df.drop(index,axis=0, inplace=True)

pandas.DataFrame.from_dict, Construct DataFrame from dict of array-like or dicts. Creates DataFrame object from dictionary by columns or by index allowing dtype specification. Parameters. Merge dataframes in a dictionary. Ask Question Asked 4 years, 11 months ago. Active 4 years, 11 months ago. Viewed 14k times 15. 5. Say I have an dictionary of

You need to assign values to dictionary keys as you iterate:

for name, df in m.items():
    m[name] = df[df['B'] > 0]

Otherwise, you're constantly overriding a variable df and not storing it anywhere.

pandas.DataFrame.to_dict, Convert the DataFrame to a dictionary. The type of the key-value pairs can be customized with the parameters (see below). Parameters. orientstr {'dict', 'list',  pandas.DataFrame.to_dict¶. Convert the DataFrame to a dictionary. The type of the key-value pairs can be customized with the parameters (see below). Determines the type of the values of the dictionary. ‘dict’ (default) : dict like {column -> {index -> value}}. ‘list’ : dict like {column -> [values]}.

If all of your dataframes have consistent indices, you should keep them together with a MultiIndex

df = pd.concat(m)

df

                      A         B         C         D
d1 2013-01-01 -0.701856  1.804441 -1.224499 -0.997452
   2013-01-02 -1.122829 -0.375963  1.476828  1.254910
   2013-01-03 -0.330781 -0.692166  1.352655 -1.296063
   2013-01-04 -0.352034  0.200128  0.411482  1.058941
   2013-01-05 -0.103345  0.119615  0.251884 -0.108792
   2013-01-06  0.690312 -1.115858 -0.271362 -0.872862
d2 2013-01-01  1.449789  0.144008 -0.445732 -0.356491
   2013-01-02  0.254142  0.102233 -0.456786  1.505599
   2013-01-03 -1.636609  0.141300 -1.458500  0.088640
   2013-01-04  0.015575  1.170128  0.229888 -0.273040
   2013-01-05  0.995011 -1.476076 -0.345353 -0.343009
   2013-01-06  0.060094  0.610622  0.192916 -1.411557

At which point you can use numerous filtering methods

df.query('B > 0')

                      A         B         C         D
d1 2013-01-01 -0.701856  1.804441 -1.224499 -0.997452
   2013-01-04 -0.352034  0.200128  0.411482  1.058941
   2013-01-05 -0.103345  0.119615  0.251884 -0.108792
d2 2013-01-01  1.449789  0.144008 -0.445732 -0.356491
   2013-01-02  0.254142  0.102233 -0.456786  1.505599
   2013-01-03 -1.636609  0.141300 -1.458500  0.088640
   2013-01-04  0.015575  1.170128  0.229888 -0.273040
   2013-01-06  0.060094  0.610622  0.192916 -1.411557

How to Convert Dictionary to Pandas DataFrame, You may use the following template to convert a dictionary to pandas DataFrame: from pandas import DataFrame my_dict = {key:value   By default orientation is columns it means keys in dictionary will be used as columns while creating DataFrame. We can also pass the orientation as ‘index’, which changes the default orientation and makes the keys in dictionary as index i.e.

Python, to_dict() method is used to convert a dataframe into a dictionary of series or list like data type depending on orient parameter. Parameters: orient: String value, ('dict', 'list', 'series', 'split', 'records', 'index') Defines which dtype to convert Columns(series into). The “orientation” of the data. If the keys of the passed dict should be the columns of the resulting DataFrame, pass ‘columns’ (default). Otherwise if the keys should be rows, pass ‘index’. Data type to force, otherwise infer.

How to convert a Pandas DataFrame into a dictionary in Python, Converting a DataFrame to a dictionary creates a mapping where each row is a key-value pair with the keys being one of the columns in the row and the values  mapper : dict-like or function Dict-like or functions transformations to apply to that axis’ values. Use either mapper and axis to specify the axis to target with mapper, or index and columns. index : dict-like or function Alternative to specifying axis ( mapper, axis=0 is equivalent to index=mapper ). columns : dict-like

Python Pandas : How to create DataFrame from dictionary , Create DataFrame from Dictionary with custom indexes. We can also pass the index list to the DataFrame constructor to replace the default  You may use the following template to convert a dictionary to pandas DataFrame: In this short tutorial, I’ll review the steps to convert a dictionary to pandas DataFrame. I’ll also share the code to create the following tool to convert your dictionary to a DataFrame: To start, gather the data for your dictionary.

Comments
  • Note iterrows is not recommended when vectorised operations are possible.
  • Agreed. Note to OP: the purpose of my solution was to show you how you could fix the solution you came up with. I would advise you use @jpp's instead
  • @jpp thanks a lot for explanation!