Dropping multiple Pandas columns by Index
I have a large pandas dataframe (>100 columns). I need to drop various sets of columns and i'm hoping there is a way of using the old
I've built selections such as:
a = df.columns[3:23] b = df.colums[-6:]
b represent column sets I want to drop.
yields the correct selection, but i can't implement it with a
ValueError: operands could not be broadcast together with shapes (20,) (6,)
I looked around but can't get my answer.
Selecting last n columns and excluding last n columns in dataframe
(Below pertains to the error I receive):
python numpy ValueError: operands could not be broadcast together with shapes
This one feels like they're having a similar issue, but the 'slices' aren't separate: Deleting multiple columns based on column names in Pandas
You can use
np.r_ to seamlessly combine multiple ranges / slices:
from string import ascii_uppercase df = pd.DataFrame(columns=list(ascii_uppercase)) idx = np.r_[3:10, -5:0] print(idx) array([ 3, 4, 5, 6, 7, 8, 9, -5, -4, -3, -2, -1])
You can then use
idx to index your columns and feed to
df.drop(df.columns[idx], axis=1, inplace=True) print(df.columns) Index(['A', 'B', 'C', 'K', 'L', 'M', 'N', 'O','P', 'Q', 'R', 'S', 'T', 'U'], dtype='object')
How to drop one or multiple columns in Pandas Dataframe , Remove columns as based on column index. Output: Method #2: Drop Columns from a Dataframe using iloc and drop() method. Remove all columns between a specific column to another columns. Let’s discuss how to drop one or multiple columns in Pandas Dataframe. Drop one or more than one columns from a DataFrame can be achieved in multiple ways. Create a
You can use this simple solution:
cols = [3,7,10,12,14,16,18,20,22] df.drop(df.columns[cols],axis=1,inplace=True)
the result :
0 1 2 4 5 6 8 9 11 13 15 17 19 21 0 3 12 10 3 2 1 7 512 64 1024.0 -1.0 -1.0 -1.0 -1.0 1 5 12 10 3 2 1 7 16 2 32.0 32.0 1024.0 -1.0 -1.0 2 5 12 10 3 2 1 7 512 2 32.0 32.0 32.0 -1.0 -1.0 3 5 12 10 3 2 1 7 16 1 32.0 64.0 1024.0 -1.0 -1.0
As you can see the columns with given index have been all deleted.
You can replace the int value by the name of the column you have in your array if we suppose you have A,B,C ...etc you can replace int values in
cols like this for example :
cols = ['A','B','C','F']
How To Drop One or More Columns in Pandas Dataframe?, How do I drop multiple columns in a data frame? How to drop column by position number from pandas Dataframe? You can find out name of first column by using this command df.columns. Indexing in python starts from 0. df.drop(df.columns, axis =1) To drop multiple columns by position (first and third columns), you can specify the position in list [0,2].
a = df.columns[3:23].values.tolist() b = df.colums[-6:].values.tolist() a.extend(b) df.drop(a,1,inplace=True)
Python Pandas : Drop columns in DataFrame by label Names or by , position 0 & 1 from dataframe object dfObj i.e. Pandas drop() is versatile and it can be used to drop rows of a dataframe as well. To use Pandas drop() function to drop columns, we provide the multiple columns that need to be dropped as a list. In addition, we also need to specify axis=1 argument to tell the drop() function that we are dropping columns.
This returns the dataframe with the columns removed
How to drop one or multiple columns in Pandas Dataframe, You can find out name of first column by using this command df. columns . Indexing in python starts from 0. To drop multiple columns by position (first and third columns), you can specify the position in list [0,2] . To drop columns by index position, we first need to find out column names from index position and then pass list of column names to drop (). For example delete columns at index position 0 & 1 from dataframe object dfObj i.e. # Delete columns at index 1 & 2 modDfObj = dfObj.drop ( [dfObj.columns  , dfObj.columns ] , axis='columns')
I have run into a similar issue before and had trouble with this but fixed it by "subtracting" one df from the other, not sure if this will work for you but it did for me:
df = df[~df.index.isin(a.index)] df = df[~df.index.isin(b.index)]
Drop multiple columns in pandas, Drop Multiple Columns by Label Names in DataFrame For example delete columns at index position 0 & 1 from dataframe object dfObj i.e.. Pandas' .drop() Method. The pandas .drop() method is used to remove entire rows or columns based on their name. If we can see that our DataFrame contains extraneous information (perhaps for example, the HR team is storing a preferred_icecream_flavor in their master records), we can destroy the column (or row) outright.
pandas.DataFrame.drop, Often while working with a bigger pandas dataframe with multiple columns, one wants to drop a column or multiple columns from a pandas DataFrame.drop(self, labels=None, axis=0, index=None, columns=None, level=None, inplace=False, errors='raise')¶. Drop specified labels from rows or columns. Remove rows or columns by specifying label names and corresponding axis, or by specifying directly index or column names.
pandas: Delete rows, columns from DataFrame with drop(), Instead of wrapping it in a list with [..], you just provide the sub selection of the columns index: df.drop(df.columns[[1, 69]], axis=1, inplace=True). Drop single and multiple columns in pandas by using column index . Drop multiple columns between two column index using iloc() function. Drop multiple columns between two column names using loc() and ix() function. Drop column name which starts with, ends with and contains a character. Drop column name in pandas using regular expression.
Delete or drop column in python pandas – drop multiple column , Remove rows or columns by specifying label names and corresponding axis, or by specifying directly index or column names. When using a multi-index, labels pandas.DataFrame.reset_index¶. Reset the index, or a level of it. Reset the index of the DataFrame, and use the default one instead. If the DataFrame has a MultiIndex, this method can remove one or more levels. Only remove the given levels from the index. Removes all levels by default. Do not try to insert index into dataframe columns.