How to shift rows up in Pandas Dataframe based on specific column

pandas shift one column
pandas groupby shift
pandas shift column one row up
pandas conditional shift
pandas lag column by group
python dataframe shift rows
pandas shift index by 1
pandas shift multiple columns

How do I shift up all the values in a row for one specific column without affecting the order of the other columns?

For example, let's say i have the following code:

import pandas as pd
data= {'ColA':["A","B","C"],

df = pd.DataFrame(data)


I would see the following output:

  ColA  ColB    ColC
0    A     0   First
1    B     1  Second
2    C     2   Third

In my case I want to verify that Column B does not have any 0s and if so, it is removed and all the other values below it get pushed up, and the order of the other columns are not affected. Presumably, I would then see the following:

  ColA  ColB    ColC
0    A   1     First
1    B   2    Second
2    C   NaN   Third

I can't figure out how to do this using either the drop() or shift() methods.

Thank you

The way I'd do this IIUC is to filter out the values in ColB which are not 0, and fill the column with these values according to the length of the obtained valid values:

m = df.loc[~df.ColB.eq(0), 'ColB'].values
df['ColB'] = float('nan')
df.loc[:m.size-1, 'ColB'] = m


 ColA  ColB    ColC
0    A   1.0   First
1    B   2.0  Second
2    C   NaN   Third

pandas.DataFrame.shift — pandas 1.1.0 documentation, When freq is not passed, shift the index without realigning the data. df.shift( periods=1, axis="columns") Col1 Col2 Col3 2020-01-01 NaN 10.0 13.0 2020-01- 02� How to shift a column in Pandas. If you want to shift your columns without re-writing the whole dataframe or you want to subtract the column value with the previous row value or if you want to find the cumulative sum without using cumsum () function or you want to shift the time index of your dataframe by Hour, Day, Week, Month or Year then to achieve all these tasks you can use pandas dataframe shift function.

Let us do simple sorted

df['ColX']=sorted(df.ColB,key=lambda x : x==invalid)
  ColA  ColB    ColC  ColX
0    A     0   First   1.0
1    B     1  Second   2.0
2    C     2   Third   NaN

Python, Pandas dataframe.shift() function Shift index by desired number of periods with an optional time freq. Creating row index values for our data frame. # We have � Pandas dataframe.shift() function Shift index by desired number of periods with an optional time freq. This function takes a scalar parameter called period, which represents the number of shifts to be made over the desired axis. This function is very helpful when dealing with time-series data. Syntax:DataFrame.shift(periods=1, freq=None, axis=0)

You can swap 0s for nans and then move up the rest of the values:

import numpy as np
df.ColB.replace(0, np.nan, inplace=True)
df.assign(ColB=df.ColB.shift(df.ColB.count() - len(df.ColB)))

How to shift a column in Pandas, DataFrame.shift (self, periods=1, freq=None, axis=0, fill_value=None) Here you can see the 0th index row value in original dataframe above is along with freq and it will shift the value based on the freq value. what does it� Utilize the shift() method for a pandas series to shift values in a column up or down. This is equivalent to LAG() or LEAD() window functions in SQL. shift() Method: Shift Values in Column Up or Down - Data Analysis

shift() Method: Shift Values in Column Up or Down, The shift() method for a pandas series helps shift values in a column up or down. out for people to ride to specific stations in the city of San Francisco. Below I show the code to replace all values in a row with NaN or NaT if� However, in this post we are going to discuss several approaches on how to drop rows from the dataframe based on certain condition applied on a column. Retain all those rows for which the applied condition on the given column evaluates to True .

shifting specific column to before/after specific column in dataframe , Stack Exchange � sign up log in python pandas data-cleaning. In dataframe If I get the question correct, you just need to change the order of your columns. This can be simply done by reassigning the new order of your columns to the dataframe. For example: The expression will vary depending on your use case. A Pandas Series function between can be used by giving the start and end date as Datetime. This is my preferred method to select rows based on dates.: df[df.datetime_col.between(start_date, end_date)] 3. Select rows between two times. Sometimes you may need to filter the rows of a DataFrame based only on time.

A Really Simple Way to Edit Row by Row in a Pandas DataFrame , Sign up and get an extra one for free. PYTHON. A Really Simple Way to Edit Row by Row in a Pandas DataFrame. Using pd.loc to change a subset of your data based on conditions. These numbers in the leftmost column are the “row indexes”, which are used to identify each row. a column_indexer , you� Often, you may want to subset a pandas dataframe based on one or more values of a specific column. Essentially, we would like to select rows based on one value or multiple values present in a column. Here are SIX examples of using Pandas dataframe to filter rows or select rows based values of a column(s).