Add Multiple Columns to Pandas Dataframe from Function

I have a pandas data frame mydf that has two columns,and both columns are datetime datatypes: mydate and mytime. I want to add three more columns: hour, weekday, and weeknum.

def getH(t): #gives the hour
    return t.hour
def getW(d): #gives the week number
    return d.isocalendar()[1] 
def getD(d): #gives the weekday
    return d.weekday() # 0 for Monday, 6 for Sunday

mydf["hour"] = mydf.apply(lambda row:getH(row["mytime"]), axis=1)
mydf["weekday"] = mydf.apply(lambda row:getD(row["mydate"]), axis=1)
mydf["weeknum"] = mydf.apply(lambda row:getW(row["mydate"]), axis=1)

The snippet works, but it's not computationally efficient as it loops through the data frame at least three times. I would just like to know if there's a faster and/or more optimal way to do this. For example, using zip or merge? If, for example, I just create one function that returns three elements, how should I implement this? To illustrate, the function would be:

def getHWd(d,t):
    return t.hour, d.isocalendar()[1], d.weekday()

Here's on approach to do it using one apply

Say, df is like

In [64]: df
Out[64]:
       mydate     mytime
0  2011-01-01 2011-11-14
1  2011-01-02 2011-11-15
2  2011-01-03 2011-11-16
3  2011-01-04 2011-11-17
4  2011-01-05 2011-11-18
5  2011-01-06 2011-11-19
6  2011-01-07 2011-11-20
7  2011-01-08 2011-11-21
8  2011-01-09 2011-11-22
9  2011-01-10 2011-11-23
10 2011-01-11 2011-11-24
11 2011-01-12 2011-11-25

We'll take the lambda function out to separate line for readability and define it like

In [65]: lambdafunc = lambda x: pd.Series([x['mytime'].hour,
                                           x['mydate'].isocalendar()[1],
                                           x['mydate'].weekday()])

And, apply and store the result to df[['hour', 'weekday', 'weeknum']]

In [66]: df[['hour', 'weekday', 'weeknum']] = df.apply(lambdafunc, axis=1)

And, the output is like

In [67]: df
Out[67]:
       mydate     mytime  hour  weekday  weeknum
0  2011-01-01 2011-11-14     0       52        5
1  2011-01-02 2011-11-15     0       52        6
2  2011-01-03 2011-11-16     0        1        0
3  2011-01-04 2011-11-17     0        1        1
4  2011-01-05 2011-11-18     0        1        2
5  2011-01-06 2011-11-19     0        1        3
6  2011-01-07 2011-11-20     0        1        4
7  2011-01-08 2011-11-21     0        1        5
8  2011-01-09 2011-11-22     0        1        6
9  2011-01-10 2011-11-23     0        2        0
10 2011-01-11 2011-11-24     0        2        1
11 2011-01-12 2011-11-25     0        2        2

How to add multiple columns to pandas dataframe in one , How to add multiple columns to pandas dataframe in one assignment? 0 votes how do i use the enumerate function inside a list? can you� In this article, I will use examples to show you how to add columns to a dataframe in Pandas. There is more than one way of adding columns to a Pandas dataframe, let’s review the main approaches. Create a Dataframe As usual let's start by creating a dataframe. Create a simple dataframe with a dictionary of lists, and column names: name, age, city, country. # Creating simple dataframe # List


To complement John Galt's answer:

Depending on the task that is performed by lambdafunc, you may experience some speedup by storing the result of apply in a new DataFrame and then joining with the original:

lambdafunc = lambda x: pd.Series([x['mytime'].hour,
                                  x['mydate'].isocalendar()[1],
                                  x['mydate'].weekday()])

newcols = df.apply(lambdafunc, axis=1)
newcols.columns = ['hour', 'weekday', 'weeknum']
newdf = df.join(newcols) 

Even if you do not see a speed improvement, I would recommend using the join. You will be able to avoid the (always annoying) SettingWithCopyWarning that may pop up when assigning directly on the columns:

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

Add New Column to Pandas DataFrame using Assign, Adding multiple columns to a DataFrame. Case 1: Add Single Column to Pandas DataFrame using Assign. To start with a simple example, let's say that you� Add new column to DataFrame in Pandas using assign() Let’s add a column ‘Marks’ i.e. mod_fd = df_obj.assign( Marks=[10, 20, 45, 33, 22, 11]) mod_fd. It will return a new dataframe with a new column ‘Marks’ in that Dataframe. Values provided in list will used as column values. Contents of new dataframe mod_fd are,


You can do this in a somewhat cleaner method by having the function you apply return a pd.Series with named elements:

def process(row):
    return pd.Series(dict(b=row["a"] * 2, c=row["a"] + 2))


my_df = pd.DataFrame(dict(a=range(10)))
new_df = my_df.join(my_df.apply(process, axis="columns"))

The result is:

   a   b   c
0  0   0   2
1  1   2   3
2  2   4   4
3  3   6   5
4  4   8   6
5  5  10   7
6  6  12   8
7  7  14   9
8  8  16  10
9  9  18  11

adding multiple columns to pandas simultaneously, For adding multiple columns you can use assign with a dict of column names and values. In [1069]: df.assign(**{'col_new_1': np.nan, 'col2_new_2': 'dogs', 'col3_new_3': 3}) Out[1069]: col_1 col_2 col2_new_2 col3_new_3 col_new_1. 0 0 4 dogs 3 NaN. 1 1 5 dogs 3 NaN. 2 2 6 dogs 3 NaN. 3 3 7 dogs 3 NaN. We can use a Python dictionary to add a new column in pandas DataFrame. Use an existing column as the key values and their respective values will be the values for new column. filter_none


def getWd(d):
    d.isocalendar()[1], d.weekday()
def getH(t):
    return t.hour
mydf["hour"] = zip(*df["mytime"].map(getH))
mydf["weekday"], mydf["weeknum"] = zip(*df["mydate"].map(getWd))

pandas.DataFrame.append — pandas 1.1.0 documentation, General function to concatenate DataFrame or Series objects. DataFrame([[5, 6 ], [7, 8]], columns=list('AB')) >>> df.append(df2) A B 0 1 2 1 3 4 0 5 6 1 7 8 DataFrames, show two ways to generate a DataFrame from multiple data sources . In general, you can use the apply function. If your function requires only one column, you can use: df['price'] = df['Symbol'].apply(getquotetoday) as @EdChum suggested. If your function requires multiple columns, you can use something like: df['new_column_name'] = df.apply(lambda x: my_function(x['value_1'], x['value_2']), axis=1) share.


adding multiple columns to pandas simultaneously, pandas add multiple columns from another dataframe I'm new to pandas and trying to figure out how to add multiple columns to pandas simultaneously. In order to add multiple columns simultaneously custom functions need to In order to� Use pd.DataFrame.apply(), as below: df['d'] = df.apply(lambda x: some_func(a = x['a'], b = x['b'], c = x['c']), axis=1) NOTE: As @ashishsingal asked about columns, the axis argument should be provided with a value of 1, as the default is 0 (as in the documentation and copied below). axis : {0 or ‘index’, 1 or ‘columns’}, default 0


Create multiple pandas DataFrame columns from applying a , I'd like to apply a function with multiple returns to a pandas DataFrame and put the results in separate new columns in that DataFrame. Apply a lambda function to all the columns in dataframe using Dataframe.apply () and inside this lambda function check if column name is ‘z’ then square all the values in it i.e. # Apply function numpy.square () to square the value one column only i.e. with column name 'z' modDfObj = dfObj.apply(lambda x: np.square(x) if x.name == 'z' else x)


Python: Add column to dataframe in Pandas ( based on other , Pandas: Add new column to Dataframe with Values in list. Suppose we want to add a Append column to dataFrame using assign() function. In Python, Pandas Library We can also add multiple columns using assign() i.e.. Original Dataframe a b c 0 222 34 23 1 333 31 11 2 444 16 21 3 555 32 22 4 666 33 27 5 777 35 11 ***** Apply a lambda function to each row or each column in Dataframe ***** *** Apply a lambda function to each column in Dataframe *** Modified Dataframe by applying lambda function on each column: a b c 0 232 44 33 1 343 41 21 2 454 26 31 3 565 42