Import Data Again

Hi Caitlin! Both double and integer are types of numbers. Integers (sometimes you can see the notations Int, In16, Int 32 or Int64) refer to whole numbers, without decimals, such as 123, and can represent variables such as years, age and - to be very niche here - sample plots in space. They can often be thought of as categorical variables. Doubles are numbers with decimal digits (like 123.5), often thought of as continuous variables, and can represent variables such as height, length, and weight.

Hi Eda! Do you want to share the exact code you used? The change in the histogram is really subtle, but you should see differences when opening the little "toggle" for more info if you define what should be read as NAs in the dataset. But you're right that changing the column type doesn't change much in this case, but I suggest that you play a bit with the arguments to see what happens in each case. What happens if you don't designate the NA values for that column? What happens if you do? What if you change the column type?

Awesome! 🎉

There are a few methods to do that! Here's an informative tutorial to inspect outliers with basic min and max functions and with plots: https://statsandr.com/blog/outliers-detection-in-r/

Plotting is one of the most effective ways to detect outliers and discrepancies - very useful in my area, where we deal with millions of biodiversity data points!

One other tool you can take a look at is OpenRefine, which can be integrated with your R scripts: https://openrefine.org/ OpenRefine is great to detect typos.

Great questions! Yes, in this case you could skip the na assignment when reading the data, and deal with it in a case-by-case level, perhaps, using mutate(), or case_when(), or na_if(). See some examples here: https://dplyr.tidyverse.org/reference/na_if.html#ref-examples

Hi Kathy! Could you share the whole code you used and the complete error message that you've got? You can also run rlang::last_trace() right after you get the error and copy-paste the message here so we have more clues about what's causing this error.

YAY we've found it! Your code has a "=" in the wrong place at line 5.

You've got:

col_types=cols(total_cup_points)="d",
harvest_year="I"))

But you should have: col_types = cols(total_cup_points = "d", harvest_year = "i")

You're closing a parenthesis after total_cup_points, which messes up a bit with the syntax of the function and makes you add an extra parenthesis after harvest_year = "I". Also, i should be lower case.

Let me know if this helps!

still getting errors. I think it is something with the input file I am going to try removing and reinstalling everything and see if that helps

still getting errors. I think it is something with the input file I am going to try removing and reinstalling everything and see if that helps

Ok! If you still get errors, feel free to copy-paste the error message and the exact code you used here. [=

Hi Maia! That's a very subtle problem: the thing is that R is trying to find the exact value "110000.00", however, although it's displayed like that in the data viewer, this number does not exist in the dataset. What exists is "110000", without the digits. You find this out when you copy-paste from the data viewer. Try this out: try copying the "110000.00" from the altitude_mean_meters column and paste it on the console or on your script. You'll see that the .00 will not paste over. Now try copying the first row value of the total_cup_points. It comes with the extra digits, right?

However, if you add these numbers to the NA list without the quotation marks, you'll get the result you want, either with or without the digits. That's because in R, 110000.00 without quotes is a numeric literal, and R stores it as the number 110000. Test this:

1.00 == 1

"1.00" == 1

You'll see R doesn't treat these values as the same thing.

In the process of reading your CSV, the na argument in read_csv() matches the raw text in the file, before any parsing happens. The function reads each cell as a string, checks whether that string is exactly one of your na values, and only then converts the rest to double, integer, and so on. That's why "11000.00" won't match what you see as 11000.00.

Long story short, you just need to remove the extra digits from your outliers (making R match the exact string value on your dataset) or remove the quotation marks around them (making R read them as a number).

I know it can be confusing! Let me know if it's not clear.

Hi Sarah! Technically, yes, but you'll need more advanced functions, some of them we'll learn on Week 2. The best approach would be to transform these values into NAs after you read the dataset, as a transformation of your raw data. Also, NA assignments should be treated carefully, so unless you are absolutely sure you will never need numbers that are smaller, and that the values are entered correctly (in some cases, 0042 can be read as a 4-digits number, but you'd like it removed), you would probably be better off just assigning specific values for your NAs. Ideally you'd have this designed into your data collection step!