Empty Rows and Columns

This lesson is called Empty Rows and Columns, part of the Data Cleaning with R course. This lesson is called Empty Rows and Columns, part of the Data Cleaning with R course.

Transcript

Click on the transcript to go to that point in the video. Please note that transcripts are auto generated and may contain minor inaccuracies.

Your Turn

Solution

## %######################################################%##
#                                                          #
####         Empty rows and columns - your turn         ####
#                                                          #
## %######################################################%##


# Import the Marine Protected Areas dataset (MPAS-your.csv)
# Identify the empty rows and columns, and create a new object with only the empty rows and columns
#  Remove the empty rows and columns

# load packages -----------------------------------------------------------
library(readr)
library(tibble)
library(dplyr)
library(naniar)
library(janitor)
library(unheadr)

# import data ---------------------------------------------------------------
MPAs <- read_csv("data/MPAS-your.csv")

# identify empty rows and columns -----------------------------------------
# empty row ids
empty_rows <- MPAs %>%
  rowid_to_column() %>%
  filter(across(c(-rowid, -`Rank (by extent)`), is.na)) %>%
  pull(rowid)
# empty column names
empty_cols <- MPAs %>%
  select(where(all_na)) %>%
  names()

# object with only empty rows and columns
MPAs %>%
  rowid_to_column() %>%
  select(rowid, empty_cols) %>%
  slice(empty_rows)


# remove empty ------------------------------------------------------------
MPAs %>% remove_empty(which = c("rows", "cols")) # still too many empty rows

MPAs %>%
  remove_empty(which = c("rows", "cols")) %>%
  mash_colnames(1) %>%
  filter(!across(-`Rank (by extent)`, is.na)) # negate the row selection

Import the Marine Protected Areas dataset (MPAS-your.csv)

Identify the empty rows and columns

Remove the empty rows and columns

Learn More

If you want to do even more with missing data, check out the naniar package.

Have any questions? Put them below and we will help you out!

You need to be signed-in to comment on this post. Login.

Alberto Cabrera • January 13, 2024

In trying to remove both rows and columns with missing data using the following Luis' solution prompted the following error.

MPA <- MPAS_your |> remove_empty(which = "cols") |> filter(!across(-rank_by_extent, is.na))

Error in filter(): ℹ In argument: !across(-rank_by_extent, is.na). Caused by error: ! ..1 must be a logical vector, not a logical matrix.

David Keyes Founder • January 15, 2024

It looks like the use of across() within filter() was deprecated in dplyr 1.0.8. You can change your code to use the if_any() function instead as follows:

MPAs %>%
  remove_empty(which = c("rows", "cols")) %>%
  mash_colnames(1) %>%
  filter(!if_any(-`Rank (by extent)`, is.na))

Alberto Cabrera • January 15, 2024

It worked. Actually I tried the version below, when I received the deprecated message. I did not realize to qualify if_any with the negation. Many thanks for sending me the correct answer.

MPAS_your |> remove_empty(which = c("rows","cols")) |> filter(if_any(-rank_by_extent,is.na))

Alberto Cabrera • January 15, 2024

It worked. Actually I tried the version below, when I received the deprecated message. I did not realize to qualify if_any with the negation. Many thanks for sending me the correct answer.

MPAS_your |> remove_empty(which = c("rows","cols")) |> filter(if_any(-rank_by_extent,is.na))