Merging a single column dataframe to another dataframe without having any common key

pandas merge on different column names
pandas join vs merge
pandas merge on index
pandas merge on multiple columns
pandas merge multiple dataframes
merging two csv files with a common column python
pandas merge columns into one
pandas merge two dataframes with different number of rows

I have dataframe as below :

                            User_Id     MARKED_CONTENT_AS_FAVOURITE RATE_CONTENT SEARCH VIEWED_CELEBRITY VIEWED_MOVIE VIEWED_TVSHOW
1 6916484f-b7bd-431a-818d-d1a63ff7c717                           0            0      1                0            4             0
2 9fbb7702-5209-46c8-b7c8-2c3d03550b56                           2            2      1                1           20             3
3 cb1fc554-8566-4c9f-a3ca-f64be302d65e                           0            0      1                1            0             0

Now I have a series with just one value. I want to perform a sort of vectorized operation where I have series with only one value.

  USER_CHECKED_IN_CONTENT
3                       0

I want to append this column to the dataframe, as follows.

                               User_Id MARKED_CONTENT_AS_FAVOURITE RATE_CONTENT SEARCH VIEWED_CELEBRITY VIEWED_MOVIE VIEWED_TVSHOW USER_CHECKED_IN_CONTENT
1 6916484f-b7bd-431a-818d-d1a63ff7c717                           0            0      1                0            4             0                       0
2 9fbb7702-5209-46c8-b7c8-2c3d03550b56                           2            2      1                1           20             3                       0
3 cb1fc554-8566-4c9f-a3ca-f64be302d65e                           0            0      1                1            0             0                       0

But when I use

pivot_activity.append(subset[[x for x in list(subset) if x not in list(pivot_activity)]]) 

It gives the output as belows :

   MARKED_CONTENT_AS_FAVOURITE  RATE_CONTENT  SEARCH  USER_CHECKED_IN_CONTENT                                User_Id  VIEWED_CELEBRITY  VIEWED_MOVIE 
0                          0.0           0.0     1.0                      NaN   6916484f-b7bd-431a-818d-d1a63ff7c717               0.0           4.0  
1                          2.0           2.0     1.0                      NaN   9fbb7702-5209-46c8-b7c8-2c3d03550b56               1.0          20.0  
2                          0.0           0.0     1.0                      NaN   cb1fc554-8566-4c9f-a3ca-f64be302d65e               1.0           0.0  
2                          NaN           NaN     NaN                      0.0                                    NaN               NaN           NaN  

you can add a new column easily to existing dataframe "df" as below:

df['USER_CHECKED_IN_CONTENT'] = 0 # df is your existing dataframe

Merging a single column dataframe to another dataframe without , you can add a new column easily to existing dataframe "df" as below: df['​USER_CHECKED_IN_CONTENT'] = 0 # df is your existing dataframe. If we directly call Dataframe.merge() on these two Dataframes, without any additional arguments, then it will merge the columns of the both the dataframes  by considering common columns as Join Keys i.e. ‘ID’ & ‘Experience’ in our case.

I think you can use concat here. Let's say you have:

df - bigger data frame
df1 - smaller data frame

You can do:

pd.concat([df, df1], axis=0).fillna(0)

Merge, join, and concatenate, The concat() function (in the main pandas namespace) does all of the heavy lifting “if any” because there is only a single possible axis of concatenation for Series. Without a little bit of context many of these arguments don't make much sense. A fairly common use of the keys argument is to override the column names  Another way to combine DataFrames is to use columns in each dataset that contain common values (a common unique id). Combining DataFrames using a common field is called "joining". The columns containing the common values are called "join key(s)".

I think this is what you are going for. A merge does not seem to be what you want since it will have to have some reference to match and you have a 1 to many scenario with differing keys. This code will assign the entire column from the subset data frame's single row single column:

pivot_activity['USER_CHECKED_IN_CONTENT'] = subset[ 'USER_CHECKED_IN_CONTENT'].head(1)

Merge, join, and concatenate, Without a little bit of context and example many of these arguments don't make much sense. A fairly common use of the keys argument is to override the column names when pandas provides a single function, merge , as the entry point for all standard Here is another example with duplicate join keys in DataFrames:. Merging Dataframe on a given column name as join key. In both the above dataframes two column names are common i.e. ID & Experience. But contents of Experience column in both the dataframes are of different types, one is int and other is string. There is no point in merging based on that column.

Merge, join, and concatenate, The concat() function (in the main pandas namespace) does all of the heavy lifting Without a little bit of context many of these arguments don't make much sense. A fairly common use of the keys argument is to override the column names when joining an index (unique) to one or more columns in a different DataFrame  You can notice that the DataFrames are now merged into a single DataFrame based on the common values present in the id column of both the DataFrames. For example, here id value 1 was present with both A , B and K , L in the DataFrame df_row hence this id got repeated twice in the final DataFrame df_merge_col with repeated value 12 of Feature3

Python| Pandas dataframe.append(), How do I append a DataFrame to another Dataframe in pandas? other DataFrame. The DataFrame to merge column-wise. func function. Function that takes two series as inputs and return a Series or a scalar. Used to merge the two dataframes column by columns. fill_value scalar value, default None. The value to fill NaNs with prior to passing any column to the merge func. overwrite bool, default True. If True, columns in self that do not exist in other will be overwritten with NaNs. Returns DataFrame. Combination of the provided DataFrames.

Joining DataFrames in Pandas, are added as new columns and the new cells are populated with NaN value. ignore_index : If True, do not use the index labels. Join and merge pandas dataframe. DataFrame (raw_data, columns = Merge two dataframes with both the left and right dataframes using the subject_id key.

Comments
  • This isn't quite a duplicate but this could be helpful: stackoverflow.com/questions/33088010/…
  • you are trying to append a single row series to a dataframe. so you will have mismatched indices. I think you need better definition of what is actually going on here and what you really want. Cause according to your question the answer below is correct.
  • Second dataframe has just one column, which has only one value. Now, I want to merge these two dataframes, with the desired output as shown.
  • I think merge is not what you want. see my answer.
  • I have two separate dataframes that I want to merge.
  • Dataframes can only be joined on some common value i.e. columns or indexes or some combination of these.