https://conorneilson.com/ Recent content on A Sky Full of Data Hugo -- gohugo.io © 2020 Fri, 13 Sep 2024 00:00:00 +0000 https://conorneilson.com/post/simulating-distribution-equivalence/ Fri, 13 Sep 2024 00:00:00 +0000 https://conorneilson.com/post/simulating-distribution-equivalence/ Been a long time since I did any simulation in R! Today I was presented with a nice opportunity too. I received a message from my brother today saying “sqrt(rand()) gives you the same probabilities as max(rand(), rand()) assuming rand() returns a number between 0 and 1.” I was immediately intrigued as this didn’t make intuitive sense to me. However, no need for intuition! This is a data-driven blog, so lets look at this in a data-driven way. https://conorneilson.com/post/converting-miles-to-kilometers-using-the-fibonacci-sequence/ Fri, 06 Sep 2024 00:00:00 +0000 https://conorneilson.com/post/converting-miles-to-kilometers-using-the-fibonacci-sequence/ Lets do a quick exploration into some interesting properties of the fibonacci sequence! Converting distances between miles and kilometers is something we’ve all done a bunch of times (maybe?). For as long as I can remember I’ve held the rough rule in my head of multiplying miles by 1.6 to get the rough amount of kilometers. This is perfectly acceptable. In 99% of day to day, this will be all you need. https://conorneilson.com/post/two-new-data-packages/ Sat, 23 Oct 2021 00:00:00 +0000 https://conorneilson.com/post/two-new-data-packages/ I’ve recently released two new data packages for R - read on to learn more about them. I’ve spent some time recently building a couple of new data packages. I was inspired by coming across a dataset by tech magazine ‘The Hustle’. They collated a dataset called “The Hustle’s iPhone vs. Android Survey” which surveys peoples phone preferences. In analysing this data I found it had a field that reported the city the respondent was from. https://conorneilson.com/post/integrating-blogdown-and-pkgdown/ Sat, 04 Sep 2021 00:00:00 +0000 https://conorneilson.com/post/integrating-blogdown-and-pkgdown/ Make your pkgdown websites be part of your blogdown website. Context I have websites set up using both blogdown and pkgdown. The reason for using both is they solve very different problems. For instance I have My main personal website, built with blogdown, and hosted by Netlify Individual websites for R packages, built with pkgdown, and hosted on Github Pages This is a case where it’s appropriate to be using both. https://conorneilson.com/post/what-i-have-in-my-rprofile/ Sun, 22 Aug 2021 00:00:00 +0000 https://conorneilson.com/post/what-i-have-in-my-rprofile/ The .Rprofile file is an extremely useful piece of R. For those who haven’t encountered .Rprofile before, it is a text file where you can put R code - this code is then source’d everytime R starts up. Thus, it makes a great place to set options you use frequently, create functions, or just to wish yourself a good morning. In this post I plan to run through the things I like to put in my . https://conorneilson.com/post/a-quieter-scrobbler/ Sun, 15 Aug 2021 00:00:00 +0000 https://conorneilson.com/post/a-quieter-scrobbler/ I just released a tiny update to scrobbler. Something you may have noticed is that scrobbler can sometimes be rather loud when you are using it. When downloading scrobbles it likes to print your username and api key periodically. This is both annoying and (in the case of the api key) rather unsafe. The new version of scrobbler will still print its progress indicators, but will no longer emit your authentication details. https://conorneilson.com/post/current-dev-project-spotty/ Sun, 15 Aug 2021 00:00:00 +0000 https://conorneilson.com/post/current-dev-project-spotty/ I started playing around with a new package today which I’ve titled spotty. spotty’s goal is to be a minimal wrapper around the spotify API. It intends to only do a small number of things but do them well. There is of course the very well done package spotifyr by Charlie Thompson. spotty is not indended to compete with this package at all, and will not come anywhere near the breadth of features offered by spotifyr https://conorneilson.com/post/generated-plots-are-now-working-by-default/ Sun, 15 Aug 2021 00:00:00 +0000 https://conorneilson.com/post/generated-plots-are-now-working-by-default/ In a previous post I described how plots generated via R code weren’t showing in my posts. The fix I ultimately found was to set a particular knitr chunk. I didn’t really understand it, but it seemed to work. This no longer seems to be the case. I tried generating some plots today to test, and they work out of the box now. Check the code below - there are no other knitr chunks in this document. https://conorneilson.com/post/scheduling-scrobbler-properly-this-time/ Sat, 14 Aug 2021 00:00:00 +0000 https://conorneilson.com/post/scheduling-scrobbler-properly-this-time/ I finally learn how Github Secrets work! My previous mistake In a previous post, I showed an example of how to use Github Actions to schedule scrobbler to run automatically and save the downloaded data into your github repo. This was a bit janky because you had to hardcode your API details into the script, and therefore it had to be a private repo in order to avoid leaking your details. https://conorneilson.com/post/scheduling-scrobbler-to-run-automatically/ Thu, 20 May 2021 00:00:00 +0000 https://conorneilson.com/post/scheduling-scrobbler-to-run-automatically/ Using github to automatically schedule, run, and store dataframes of your scrobbles in a repository. One of the biggest issues you can run into when trying to analyse your scrobble data is waiting for it all to download when running download_scrobbles(). While the addition of update_scrobbles() partially helps this, it still relies on you having a relatively recent source of scrobbles available in a dataset. The idea behind this post is to setup a github repo where you can store scrobbled datasets. https://conorneilson.com/post/announcing-words-an-r-package-containing-scrabble-allowed-english-words/index.md/ Sun, 16 May 2021 00:00:00 +0000 https://conorneilson.com/post/announcing-words-an-r-package-containing-scrabble-allowed-english-words/index.md/ I’m thrilled to announce the release of my new package words. words is an extremely simple package, but I’m excited to use it for my future projects. Recently I’ve been playing around with some word based puzzles in R. One of the examples of this is wordbuilder, which finds all valid words of from a given list of 5 letters. In an example like this, one of the challenges is finding a list of valid english words. https://conorneilson.com/post/generated-plots-not-showing-in-blogdown/ Sun, 04 Apr 2021 00:00:00 +0000 https://conorneilson.com/post/generated-plots-not-showing-in-blogdown/ Debugging why my plots weren’t showing up in blogdown While writing my last post, I discovered that while using blogdown::serve_site(), any plots generated by R code weren’t showing up in the website. The first thing I did was remove the .gitignore file under my static directory. This unfortunately didn’t fix the issue (but I suspect was still a contributor to the final fix.) The second thing I did was add a setup R chunk that had the code knitr::opts_chunk$set(echo = TRUE, fig. https://conorneilson.com/post/auto-authentication-with-scrobbler/ Sun, 31 Jan 2021 00:00:00 +0000 https://conorneilson.com/post/auto-authentication-with-scrobbler/ Authenticate automatically using environment variables I’m pleased to announce a new version of scrobbler which introduces a slightly new way to authenticate download_scrobbles and update_scrobbles calls. This change is released on CRAN, so you can get the update by running install.packages("scrobbler") New authentication In previous versions of scrobbler, whenever you wanted to download scrobbles you had to pass in your username and API key. library(scrobbler) my_data <- download_scrobbles(username = "your_username", api_key = "your_api_key") This quickly gets tiring (particularly with the API key), so I’m pleased to announce there is now a better way. https://conorneilson.com/post/tracking-your-music-history-with-scrobbler-part-2/ Fri, 15 Jan 2021 00:00:00 +0000 https://conorneilson.com/post/tracking-your-music-history-with-scrobbler-part-2/ In part 2 of this series, we’ll be looking at how to actually start using scrobbler to download your music history. As a quick recap from part 1, scrobbling is a service offered by Last.fm, where any music you listen to on other platforms (i.e., Spotify) is recorded and stored by Last.fm, along with a timestamp and associated metadata. scrobbler is intended as an interface to that database, so you can download and explore your music history with the full freedom of R. https://conorneilson.com/post/setting-your-posts-preview-in-blogdown/ Wed, 23 Dec 2020 00:00:00 +0000 https://conorneilson.com/post/setting-your-posts-preview-in-blogdown/ In which I learn about post summaries. In various hugo themes, you get the option of having a ‘preview’ of the text that your post contains. For example, this is how it looks on my blog: I’ve always been irritated by not being able to control what text went into that little description section. Early in my blogdown journey I looked into it, but quickly got lost in Hugo’s .Summary and . https://conorneilson.com/post/finding-my-most-played-song-in-a-single-day-with-scrobbler/ Thu, 17 Dec 2020 00:00:00 +0000 https://conorneilson.com/post/finding-my-most-played-song-in-a-single-day-with-scrobbler/ Ever since publishing my first post about the development of scrobbler I’ve been meaning to spend much more time writing and showcasing some of the analyses I wanted to do with it. As with many things, life gets in the way, but this is my effort to show something small and simple. My idea for this post came from Spotify’s ‘2020 Wrapped’ video they make for you showing trends of your listening over the previous year. https://conorneilson.com/post/tracking-your-music-history-with-scrobbler/ Mon, 11 May 2020 00:00:00 +0000 https://conorneilson.com/post/tracking-your-music-history-with-scrobbler/ Note: This is the first of a 2 part post about scrobbler. This first one will deal with what exactly scrobbles are and how to get started, while part 2 will focus on actually using the scrobbler R package. Like many people, I’ve always loved music. While my skills at playing music are embarrassingly non-existent, I basically listen to music all day every day. Therefore, it was only inevitable that I would eventually start to muse on how I could apply my love of data to my love of music… https://conorneilson.com/post/resetting-fileinput-in-shiny/ Sun, 05 Apr 2020 00:00:00 +0000 https://conorneilson.com/post/resetting-fileinput-in-shiny/ While working on my (currently WIP) package nebula, I ran into an issue with resetting the fileInput component in Shiny. In short, nebula allows you to upload a photo of an otolith (fish ear bone), and mark the positions of the growth rings (like in a tree trunk). Once you’ve finished, you can download the positions as a csv file. The issue I ran into, is once you’ve finished marking your photo, if you want to then start a new photo, you need some way of resetting the fileInput, and removing the current photo. https://conorneilson.com/post/a-silly-typing-mistake/ Sat, 14 Mar 2020 00:00:00 +0000 https://conorneilson.com/post/a-silly-typing-mistake/ Recently I was working on my WIP package ‘verhoeff’, for calculating check digits based on the Verhoeff algorithim. I was at the stage where most of the basic code was written, so I was starting to test simple examples. I was testing an early implementation which I thought would work. I used the code calculate_digit(prepare_number(1), dat$d5, dat$d5_p, dat$inv_v) In this code, prepare_number simply does some checks to see if the number is valid, and converts it to a numeric vector. https://conorneilson.com/post/12-days-of-christmas/ Sat, 07 Mar 2020 00:00:00 +0000 https://conorneilson.com/post/12-days-of-christmas/ Over the holiday period my partner was talking about a question she was posed at a work quiz, which was “How many gifts in total are given in the song”The 12 Days of Christmas”. I thought this was a great opportunity to showcase some of the basic skills that are useful to master in R, namely functions and *apply. First, we need to consider how to calculate the number of presents gifted in any given day. https://conorneilson.com/about/ Mon, 01 Jan 0001 00:00:00 +0000 https://conorneilson.com/about/ I was born and raised in New Zealand, in a thriving metropolis of 2000 people (which, incidentally, was a Lord of the Rings filming location). When I was at high school I decided I wanted to be a Marine Biologist. A teacher at the time told me I should do a double major with Statistics, since a Marine Biology degree on its own wouldn’t be very valuable. I was pretty against this; Math wasn’t my strong subject and I was loathe to sign up for three years of college level maths. https://conorneilson.com/misc/ Mon, 01 Jan 0001 00:00:00 +0000 https://conorneilson.com/misc/ A miscellaneous collection of things that I like. Books Lord of the Rings (and anything Tolkien) The Kingkiller Chronicles by Patrick Rothfuss Harry Potter Cats Cradle Shantaram Looking for Alaska Jodi Picoult (particularly The Tenth Circle) Gardens of the Moon Music Shania Twain Cassadee Pope Sheryl Crow Bruce Springsteen The Corrs Bon Jovi Movies Lord of the Rings Black Swan Blade Runner 2049 The Dark Knight The Last Samurai https://conorneilson.com/r/ Mon, 01 Jan 0001 00:00:00 +0000 https://conorneilson.com/r/ I’ve written several R packages, mostly for my own learning and enjoyment. I started writing packages when I finished my MSc, and kinda just kept on doing it here and there. This is a list of everything I consider it be either in a useable state, or is being actively developed on. scrobbler: Download ‘Scrobbles’ from ‘Last.fm’ fishdata: A Small Collection of Fish Population Datasets verhoeff: Implementation of the ‘Verhoeff’ Check Digit Algorithm https://conorneilson.com/reading/ Mon, 01 Jan 0001 00:00:00 +0000 https://conorneilson.com/reading/ This is a collection of various readings I like. Rather than novels, this list will focus more on things like blog posts, tutorials, etc, and will mainly be about programming/statistics/tech. dplyr vs data.table - a community post on Stack Overflow discussing dplyr and data.table. Why I don’t use ggplot2 by Jeff Leek - Another piece that I totally disagree with, but think it’s a very good piece of writing. https://conorneilson.com/thesis/ Mon, 01 Jan 0001 00:00:00 +0000 https://conorneilson.com/thesis/ I wrote a Masters thesis in 2016 about the life history, migration patterns, and developmental strategies of a New Zealand fish species. I host a copy of it as a gitbook here. and here Note: The version linked above is not the official copy I submitted to Victoria University of Wellington. There have been some minor changes as I updated it for gitbook format, and thus there may be some formatting inconsistencies.