Column referencing after dplyr doesn't work

dplyr mutate_at
dplyr add column
mutate_if
transform column dplyr
dplyr : : cheat sheet
dplyr select
mutate if
dplyr vars

This question already has answers here:

The dplyr package is part of the tidyverse which is built around a modified version of the data.frame called a tibble which behaves slightly different from a normal data.frame.

class(temp)
[1] "data.frame"

temp2 = temp %>% group_by(gear) %>% mutate(var.wt = var(wt)) %>% ungroup()
class(temp2)
[1] "tbl_df"     "tbl"        "data.frame"

One difference is that when you subset a single column of a tibble, the result remains a tibble, rather than being converted to a vector as with a data.frame:

temp[,'wt']
 [1] 2.620 2.875 2.320 3.215 3.440 3.460 3.570 3.190 3.150 3.440 3.440 4.070 3.730 3.780 5.250 5.424 5.345 2.200 1.615 1.835 2.465 3.520 3.435 3.840
[25] 3.845 1.935 2.140 1.513 3.170 2.770 3.570 2.780

temp2[,'wt']
# A tibble: 32 x 1
      wt
   <dbl>
 1  2.62
 2  2.88
 3  2.32
 4  3.22
 5  3.44
 6  3.46
 7  3.57
 8  3.19
 9  3.15
10  3.44
# ... with 22 more rows

Since mean expects to act on a vector, it returns an error when you use it with a tibble. You can either use as.data.frame to convert it back to a data.frame:

temp3 <- as.data.frame(temp2)
class(temp3)
[1] "data.frame"

mean(temp3[,'wt'])
[1] 3.21725

Or subset with $ or double brackets [[ which both return vectors:

temp2$wt
 [1] 2.620 2.875 2.320 3.215 3.440 3.460 3.570 3.190 3.150 3.440 3.440 4.070 3.730 3.780 5.250 5.424 5.345 2.200 1.615 1.835 2.465 3.520 3.435 3.840
[25] 3.845 1.935 2.140 1.513 3.170 2.770 3.570 2.780

mean(temp2$wt)
[1] 3.21725

mean(temp2[['wt']])
[1] 3.21725

dplyr: how to reference columns by column index rather than column , You can try: iris %>% head %>% mutate(sum = .[[1]] + .[[2]]) Sepal.Length Sepal.Width Petal.Length Petal.Width Species sum 1 5.1 3.5 1.4 0.2 Let’s try to modify DepTime column name to DepartureTime by using r dplyr rename column. Verify the column names after applying the dplyr rename() function. Remember that unless you save the changes back to a variable , the changes made to a dataframe using dplyr operations doesn’t come into effect. So, if you want the renamed column name to be applied to your tibble, you will need to save it back to a variable again.

Function reference • dplyr, all_equal() all.equal(<tbl_df>). Flexible equality comparison for data frames. bind_rows() bind_cols(). Efficiently bind multiple data frames by row and column. Selecting columns and filtering rows. We’re going to learn some of the most common dplyr functions: select(), filter(), mutate(), group_by(), and summarize(). To select columns of a data frame, use select(). The first argument to this function is the data frame (metadata), and the subsequent arguments are the columns to keep.

There are other people that have pointed out that your class has changed and this is why you are getting a problem. But there are reasons why some classes (including tibble in dplyr) do things the way they do. And understanding will help you build more robust code in the future.

Let's look at some objects and their classes.

Start with the mtcars dataset. It's a data.frame.

temp = mtcars
class(temp)
# [1] "data.frame"

When you subset it using the default square brackets you get a numeric vector.

temp2 <- temp[,'wt']
class(temp2)
# [1] "numeric"

When you do some work on the mtcars data using dplyr you get a tibble (aka tbl) back out.

temp3 <- group_by(gear) %>% mutate(var.wt = var(wt)) %>% ungroup()
class(temp3)
# [1] "tbl_df"     "tbl"        "data.frame"

When you try to subset this tibble you get another tibble!!!

class(temp3[,"wt"])
# [1] "tbl_df"     "tbl"        "data.frame"

But WHY!? Well, the answer is that tibbles assume that you always want a tibble back. Dataframes assume that you want a dataframe back unless there is only one column selected. If you are programming through an arbritrary number of columns this is a good thing because your code will always perform the same.

There are two ways to get a tibble to return a column. First is to use the $ notation.

class(temp3$wt)
# [1] "numeric"

The other option is to use the drop = TRUE option. This will change the default behavior of the tibble and make it drop everything but the vector.

class(temp3[,"wt",drop = TRUE])
# [1] "numeric"

Create or transform variables, mutate() does not evaluate the expressions when the group is empty. To preserve, convert to an explicit variable with tibble::rownames_to_column() . Dynamic column/variable names with dplyr using Standard Evaluation functions. Data manipulation works like a charm in R when using a library like dplyr. An often overlooked feature of this library is called Standard Evaluation (SE) which is also described in the vignette about the related Non-standard Evaluation.

Mutate multiple columns, x. a vector of values. n. a positive integer of length 1, giving the number of positions to lead or lag by. default. value used for non-existent rows. Defaults to NA . Note the underscore at the end of group_by_, this means that we are using the standard evaluation version of group_by, therefore we need to pass the column name with quotes. Looking at the manual of any dplyr function, you will see there is always a standard evaluation version of the usual functions,

Lead and lag., A list of columns generated by vars() , a character vector of column names, a numeric vector of column positions, or NULL . .predicate. A predicate function to be Dynamic column/variable names with dplyr using Standard Evaluation functions. Data manipulation works like a charm in R when using a library like dplyr. An often overlooked feature of this library is called Standard Evaluation (SE) which is also described in the vignette about the related Non-standard Evaluation.

Operate on a selection of variables, This argument is automatically quoted and later evaluated in the context inspired by the SQL COALESCE function which does the same thing for NULLs. Values in list columns are compared by reference, this can lead to. It seems like dplyr::filter is behaving differently; at least some older code is not working the way that it used to. Often I use the Interval class from lubridate in my work and usually an interval column in a tbl_df doesn't throw filter off in this way, take a look: