fprog.qmd

# Functional programming

Now that we are familiar with Git and Github, we can start with writing code. We
will learn how to write code using the functional programming paradigm.
Programming paradigms are ways to structure programs (or scripts).

This chapter will teach you the fundamentals of functional programming.
*Functional programming* might sound scary, but we will focus on only a handful
of concepts that are quite accessible but still provide many benefits. Using
these functional programming concepts will make your code more reliable, easier
to test, document, share, and ultimately rerun.

## Introduction

Remember that the philosophy of part one of this book is "don't repeat
yourself". In this chapter we will see how we can reduce the amount of code as
much as possible. In the previous chapter we’ve seen how Bruno was able to get
rid of many lines of code (that were all the same) by writing a single function:

```{r, eval = F}
make_plot <- function(country_level_data,
                      commune_level_data,
                      commune){

  filtered_data <- commune_level_data %>%
    filter(locality == commune)

  data_to_plot <- bind_rows(
    country_level_data,
    filtered_data
  )

  ggplot(data_to_plot) +
    geom_line(aes(y = pl_m2,
                  x = year,
                  group = locality,
                  colour = locality))
}

```

Now we are going to go one step further and not only learn how to write good
functions, but also how we can push the concept of "not repeating oneself" to
the extreme by using higher-order functions and function factories.

You are very likely already familiar with at least two elements of functional
programming: functions and lists. But functional programming is a complete
programming paradigm, so using functional programming is more than simply using
functions and lists (which you can use with other programming paradigms as
well).

Functional programming is a paradigm that relies exclusively on the evaluation
of functions to achieve the desired result. If you have already written your own
functions in the past, what follows will not be very new. But in order to write
a good functional program, the functions that you write and evaluate have to
have certain properties. Before discussing these properties, let's start with
*state*.

### The state of your program

Let's suppose that you start a fresh R session, and immediately run this line:

```{r, eval = F}
ls()
```

If you did not modify any of R's configuration files that get automatically
loaded on startup, you should see the following:

```{r, eval = F}
character(0)
```

Let's suppose that now you load some data:

```{r, eval = F}
data(mtcars)
```

and define a variable `a`:

```{r, eval = F}
a <- 1
```

Running `ls()` now shows the following:

```{r, eval = F}
[1] "a"      "mtcars"
```

You have just altered the state of your program. You can think of the *state* as
a box that holds everything that gets defined by the user and is accessible at
any time. Let's now define a simple function that prints a sentence:

```{r, eval = F}
f <- function(name){
  print(paste0(name, " likes lasagna"))
}

f("Bruno")
```

and here's the output:

```{r, eval = F}
[1] "Bruno likes lasagna"
```

Let's run `ls()` again:

```{r, eval = F}
[1] "a"      "f"      "mtcars"
```

Function `f()` is now listed there as well. This function has two nice properties:

- For a given input, it always returns exactly the same output. So `f("Bruno")` will always return "Bruno likes lasagna".
- When running this function, the state of the program does not get altered in any way.

### Predictable functions

Let's now define another function called `g()`, which does not have the same
properties as `f()`. First, let's define a function which does not always return
the same output given a particular input:

```{r, eval = F}
g <- function(name){
  food <- sample(c("lasagna", "cassoulet", "feijoada"), 1)
  print(paste0(name, " likes ", food))
}
```

For the same input, "Bruno", this function now produces (potentially) a
different output:

```{r, eval = F}
g("Bruno")
[1] "Bruno likes lasagna"
```

```{r, eval = F}
g("Bruno")
[1] "Bruno likes feijoada"
```

And now let's consider function `h()` that modifies the state of the program:

```{r, eval = F}
h <- function(name){
  food <- sample(c("lasagna", "cassoulet", "feijoada"), 1)

  if(exists("food_list")){
    food_list <<- append(food_list, food)
  } else {
    food_list <<- append(list(), food)
  }

  print(paste0(name, " likes ", food))
}
```

This function uses the `<<-` operator. This operator saves definitions that are
made inside the body of functions (the body of a function are all the
instructions that go between the curly braces) in the global environment.
Before calling this function, run `ls()` again. You should see the same objects
as before, plus the new functions we've defined:

```{r, eval = F}
[1] "a"       "f"        "g"         "h"         "mtcars"
```

Let's now run `h()` once:

```{r, eval = F}
h("Bruno")
[1] "Bruno likes feijoada"
```

And now `ls()` again:

```{r, eval = F}
[1] "a"       "f"       "food_list" "g"       "h"       "mtcars"
```

Running `h()` did two things: it printed the message, but also created a
variable called "food_list" in the global environment with the following
contents:

```{r, eval = F}
food_list
```

```{r, eval = F}
[[1]]
[1] "feijoada"
```

Let's run `h()` again:

```{r, eval = F}
h("Bruno")
[1] "Bruno likes cassoulet"
```

and let's check the contents of "food_list":

```{r, eval = F}
food_list
```

```{r, eval = F}
[[1]]
[1] "feijoada"

[[2]]
[1] "cassoulet"
```

If you keep running `h()`, this list will continue growing. Let me say that I
hesitated to show you this; this is because if you didn't know `<<-`, you might
find the example above useful. But while useful, it is quite dangerous as well.
Generally, we want to avoid using functions that change the state as much as
possible because these functions are unpredictable, especially if randomness is
involved. It is much safer to define `h()` like this instead:

```{r, eval = F}
h <- function(name, food_list = list()){

  food <- sample(c("lasagna", "cassoulet", "feijoada"), 1)

  food_list <- append(food_list, food)

  print(paste0(name, " likes ", food))

  food_list
}
```

The difference now is that we made `food_list` the second argument of the
function. Also, we defined it as being optional by writing:

```{r, eval = F}
food_list = list()
```

This means that if we omit this argument, the empty list will get used by
default. This avoids the users from having to manually specify it.

We can call it like this:

```{r, eval = F}
food_list <- h("Bruno", food_list)
 # since food_list is
 # already defined,  we don't
 # need to start with an empty list
```

```{r, eval = F}
[1] "Bruno likes feijoada"
```

We save the output back to `food_list`. Let's now check its contents:

```{r, eval = F}
food_list
```

```{r, eval = F}
[[1]]
[1] "feijoada"

[[2]]
[1] "cassoulet"

[[3]]
[1] "feijoada"
```

The only thing that we now still need to deal with is the fact that the food
item gets chosen randomly. I'm going to show you the simple way of dealing with
this, but later in this chapter we are going to use the `{withr}` package for
situations like this. Let's redefine `h()` one last time:

```{r, eval = F}
h <- function(name, food_list = list(), seed = 123){

  # We set the seed, making sure that we get
  # the same selection of food for a given seed

  set.seed(seed)
  food <- sample(c("lasagna", "cassoulet", "feijoada"), 1)

  # We now need to unset the seed, because
  # if we don't, guess what, the seed will
  # stay set for the whole session!

  set.seed(NULL)

  food_list <- append(food_list, food)

  print(paste0(name, " likes ", food))

  food_list
}
```

Let's now call `h()` several times with its default arguments:

```{r, eval = F}
h("Bruno")
```

```{r, eval = F}
[1] "Bruno likes feijoada"
[[1]]
[1] "feijoada"
```

```{r, eval = F}
h("Bruno")
```

```{r, eval = F}
[1] "Bruno likes feijoada"
[[1]]
[1] "feijoada"
```

```{r, eval = F}
h("Bruno")
```

```{r, eval = F}
[1] "Bruno likes feijoada"
[[1]]
[1] "feijoada"
```

As you can see, every time this function runs, it now outputs the same result.
Users can change the seed to have this function output, consistently, another
result.

### Referentially transparent and pure functions

A referentially transparent function is a function that does not use any
variable that is not also one of its inputs. For example, the following
function:

```{r, eval = F}
bad <- function(x){
  x + y
}
```

is not referentially transparent, because `y` is not one of the function’s
inputs. What happens if you run `bad()` is that `bad()` needs to look for `y`.
Because `y` is not one of its inputs, `bad()` then looks for it in the global
environment. If `y` is defined there, it then gets used. Defining and using such
functions must be avoided at all costs because these functions are
unpredictable. For example:


```{r, eval = F}
y <- 10

bad <- function(x){
  x + y
}

bad(5)
```

This will return `15`. But if `y <- 45` then `bad(5)` would this time around
return `50`. It is much safer, and clearer to make `y` an explicit input of the
function instead of having to keep track of `y`’s value (and it's so easy to do,
why just not do it):

```{r, eval = F}
good <- function(x, y){
  x + y
}
```

`good()` is a referentially transparent function; it is much safer than `bad()`.
`good()` is also a pure function, because it's a function that does not interact
in any way with the global environment. It does not write anything to the global
environment, nor requires anything from the global environment. Function `h()`
from the previous section was not pure, because it created an object and wrote
it to the global environment (the `food_list` object). Turns out that pure
functions are thus necessarily referentially transparent.

So the first lesson in your functional programming journey that you have to
remember is to only use pure functions.

## Writing good functions

### Functions are first-class objects

In a functional programming language, functions are first-class objects.
Contrary to what the name implies, this means that functions, especially the
ones you define yourself, are nothing special. A function is an object like any
other, and can thus be manipulated as such. Think of anything that you can do
with any object in R, and you can do the same thing with a function. For
example, let's consider the `+()` function. It takes two numeric objects and
returns their sum:

```{r}
1 + 5.3

# or alternatively: `+`(1, 5.3)
```

You can replace the numbers with functions that return numbers:

```{r}
sqrt(1) + log(5.3)
```

It's also possible to define a function that explicitly takes another function
as an input:

```{r}
h <- function(number, f){
  f(number)
}
```

You can call then use `h()` as a wrapper for `f()`:

```{r}
h(4, sqrt)

h(10, log10)
```

Because `h()` takes another function as an argument, `h()` is called a
higher-order function.

If you don't know how many arguments `f()`, the function you're wrapping, has,
you can use the `...`:

```{r}
h <- function(number, f, ...){
  f(number, ...)
}
```

`...` are simply a place-holder for any potential additional argument that `f()`
might have:

```{r}
h(c(1, 2, NA, 3), mean, na.rm = TRUE)

h(c(1, 2, NA, 3), mean, na.rm = FALSE)
```

`na.rm` is an argument of `mean()`. As the developer of `h()`, I don't
necessarily know what `f()` might be, but even if I knew what `f()` would be and
knew all its arguments, I might not want to list them all. So I can use `...`
instead. The following is also possible:

```{r}
w <- function(...){
  paste0("First argument: ", ..1,
         ", second argument: ", ..2,
         ", last argument: ", ..3)
}

w(1, 2, 3)
```

If you want to learn more about `...`, type `?dots` in an R console.

Because functions are nothing special, you can also write functions that return
functions. As an illustration, we'll be writing a function that converts
warnings to errors. This can be quite useful if you want your functions to fail
early, which often makes debugging easier. For example, try running this:

```{r}
sqrt(-5)
```

This only raises a warning and returns `NaN` (Not a Number). This can be quite
dangerous, especially when working non-interactively, which is what we will be
doing a lot later on. It is much better if a pipeline fails early due to an
error, than dragging a `NaN` value. This also happens with `log10()`:

```{r}
log10(-10)
```

So it could be useful to redefine these functions to raise an error instead, for
example like this:

```{r}
strict_sqrt <- function(x){

  if(x < 0) stop("x is negative")

  sqrt(x)

}
```

This function now throws an error for negative `x`:

```{r, eval = F}
strict_sqrt(-10)
```

```
Error in strict_sqrt(-10) : x is negative
```

However, it can be quite tedious to redefine every function that we need in our
pipeline, and remember, we don't want to repeat ourselves. So, because functions
are nothing special, we can define a function that takes a function as an
argument, converts any warning thrown by that function into an error, and
returns a new function. For example:

```{r}
strictly <- function(f){
  function(...){
    tryCatch({
      f(...)
    },
    warning = function(warning)stop("Can't do that chief"))
  }
}

```

This function makes use of `tryCatch()` which catches warnings raised by an
expression (in this example the expression is `f(...)`) and then raises an error
instead with the `stop()` function. It is now possible to define new functions
like this:

```{r}
s_sqrt <- strictly(sqrt)
```

```{r, eval = FALSE}
s_sqrt(-4)
```

```
Error in value[[3L]](cond) : Can't do that chief
```

```{r}
s_log <- strictly(log)
```

```{r, eval = FALSE}
s_log(-4)
```

```
Error in value[[3L]](cond) : Can't do that chief
```

Functions that return functions are called *function factories* and they're
incredibly useful. I use this so much that I've written a package, available on
CRAN, called `{chronicler}`, that does this:

```{r}
s_sqrt <- chronicler::record(sqrt)
```

```{r}
result <- s_sqrt(-4)

result
```

Because the expression above resulted in an error, `Nothing` is returned.
`Nothing` is a special value defined in the `{maybe}` package (check it out,
a very interesting package!). We can then even read a log to see what went
wrong:

```{r}
chronicler::read_log(result)
```

The `{purrr}` package also comes with function factories that you might find
useful (`{possibly}`, `{safely}` and `{quietly}`).

In part 2 we will also learn about assertive programming, another way of making
our functions safer, as an alternative to using function factories.

### Optional arguments

It is possible to make functions’ arguments optional, by using `NULL`. For
example:

```{r}
g <- function(x, y = NULL){
  if(is.null(y)){
    print("optional argument y is NULL")
    x
  } else {
    if(y == 5) print("y is present"); x+y
  }
}
```

Calling `g(10)` prints the message "Optional argument y is NULL", and
returns 10. Calling `g(10, 5)` however, prints "y is present" and returns 15. It
is also possible to use `missing()`:

```{r}
g <- function(x, y){
  if(missing(y)){
    print("optional argument y is missing")
    x
  } else {
    if(y == 5) print("y is present"); x+y
  }
}
```

I however prefer the first approach, because it is clearer which arguments are
optional, which is not the case with the second approach, where you need to read
the body of the function.

### Safe functions

It is important that your functions are safe and predictable. You should avoid
writing functions that behave like the `nchar()` base function. Let's see why
this function is not safe:

```{r}
nchar("10000000")
```

It returns the expected result of 8. But what if I remove the quotes?

```{r}
nchar(10000000)
```

What is going on here? I'll give you a hint: simply type `10000000` in the
console:

```{r}
10000000
```

`10000000` gets represented as `1e+07` by R. This number in scientific notation
gets then converted into the character "1e+07" by `nchar()`, and this conversion
happens silently. `nchar()` then counts the number of characters, and
*correctly* returns 5. The problem is that it doesn't make sense to provide a
number to a function that expects a character. This function should have
returned an error message, or at the very least raised a warning that the number
got converted into a character. Here is how you could rewrite `nchar()` to make
it safer:

```{r}
nchar2 <- function(x, result = 0){

  if(!isTRUE(is.character(x))){
    stop(paste0("x should be of type 'character', but is of type '",
                typeof(x), "' instead."))
  } else if(x == ""){
    result
  } else {
    result <- result + 1
    split_x <- strsplit(x, split = "")[[1]]
    nchar2(paste0(split_x[-1],
                     collapse = ""), result)
  }
}
```

This function now returns an error message if the input is not a character:

```{r, eval = F}
nchar2(10000000)
```

```
Error in nchar2(10000000) : x should be of type 'character', but is of type 'integer' instead.
```

This section is in a sense an introduction to assertive programming. As
mentioned in the section on function factories, we will be learning about
assertive programming in greater detail in part 2 of the book.

### Recursive functions

You may have noticed in the last lines of `nchar2()` (defined above) that
`nchar2()` calls itself. A function that calls itself in its own body is called
a recursive function. It is sometimes easier to define a function in its
recursive form than in an iterative form. The most common example is the
factorial function. However, there is an issue with recursive functions (in the
R programming language, other programming languages may not have the same
problem, like Haskell): while it is sometimes easier to write a function using a
recursive algorithm than an iterative algorithm, like for the factorial
function, recursive functions in R are quite slow. Let's take a look at two
definitions of the factorial function, one recursive, the other iterative:

```{r}
fact_iter <- function(n){
  result = 1
  for(i in 1:n){
    result = result * i
  }
  result
}

fact_recur <- function(n){
  if(n == 0 || n == 1){
  result = 1
  } else {
    n * fact_recur(n-1)
  }
}
```

Using the `{microbenchmark}` package we can benchmark the code:

```{r, eval = FALSE}
microbenchmark::microbenchmark(
  fact_recur(50),
  fact_iter(50)
)
```

```
Unit: microseconds
           expr    min     lq     mean median      uq    max neval
 fact_recur(50) 21.501 21.701 23.82701 21.901 22.0515 68.902   100
  fact_iter(50)  2.000  2.101  2.74599  2.201  2.3510 21.000   100
```

We see that the recursive factorial function is 10 times slower than the
iterative version. In this particular example it doesn't make much of a
difference, because the functions only take microseconds to run. But if you're
working with more complex functions, this is a problem. If you want to keep
using the recursive function and not switch to an iterative algorithm, there are
ways to make them faster. The first is called *trampolining*. I won't go into
details, but if you're interested, there is an R package that allows you to use
trampolining with R, aptly called
[`{trampoline}`](https://rdinnager.github.io/trampoline/)^[https://rdinnager.github.io/trampoline/].
Another solution is using the [`{memoise}`](https://memoise.r-lib.org/)^[https://memoise.r-lib.org/] package.
Again, I won't go into details. So if you want to use and optimize recursive
functions, take a look at these packages.

### Anonymous functions

It is possible to define a function and not give it a name. For example:

```{r, eval = F}
function(x)(x+1)(10)
```

Since R version 4.1, there is even a shorthand notation for anonymous functions:

```{r, eval = F}
(\(x)(x+1))(10)
```

Because we don't name them, we cannot reuse them. So why is this useful?
Anonymous functions are useful when you need to apply a function somewhere
inside a pipe once, and don't want to define a function just for this. This will
become clearer once we learn about lists, but before that, let's philosophize a
bit.

### The Unix philosophy applied to R

>This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

*Doug McIlroy, in A Quarter Century of Unix*[^1]

[^1]: http://www.catb.org/~esr/writings/taoup/html/ch01s06.html

We can take inspiration from the Unix philosophy and rewrite it for our
purposes:

*Write functions that do one thing and do it well. Write functions that work together. Write functions that handle lists, because that is a universal interface.*

Strive for writing simple functions that only perform one task. Don’t hesitate
to split a big function into smaller ones. Small functions that only perform one
task are easier to maintain, test, document and debug. These smaller functions
can then be chained using the `|>` operator. In other words, it is preferable to
have something like:

```
a |> f() |> g() |> h()
```

where `a` is for example a path to a data set, and where `f()`, `g()` and `h()`
successively read, clean, and plot the data, than having something like:

```
big_function(a)
```

that does all the steps above in one go.

This idea of splitting the problem into smaller chunks, each chunk in turn split
into even smaller units that can be handled by functions and then the results of
these function combined into a final output is called composition.

The advantage of splitting `big_function()` into `f()`, `g()` and `h()` is that
you can *eat the elephant one bite at a time*, and also reuse these smaller
functions in other projects more easily. So what’s important is that you can
make small functions work together by sharing a common interface. The list is
usually a good candidate for this.

## Lists: a powerful data-structure

Lists are the second important ingredient of functional programming. In the R
philosophy inspired by the UNIX philosophy, I stated that *lists are a universal
interface* in R, so our functions should handle lists. This of course depends on
what it is you’re doing. If you need functions to handle numbers, then there's
little value in placing these numbers inside lists. But in practice, you will
very likely manipulate objects that are more complex than numbers, and this is
where lists come into play.

### Lists all the way down

Lists are extremely flexible, and most of the very complex objects classes that
you manipulate are actually lists, but just fancier. For example, a data frame
is a list:

```{r}
data(mtcars)

typeof(mtcars)
```

A fitted model is a list:

```{r}
my_model <- lm(hp ~ mpg, data = mtcars)

typeof(my_model)
```

A `ggplot` is a list:

```{r}
library(ggplot2)

my_plot <- ggplot(data = mtcars) +
  geom_line(aes(y = hp, x = mpg))

typeof(my_plot)
```

It's lists all the way down, and it's not a coincidence; it’s because lists are
very powerful. So it's important to know what you can do with lists.

### Lists can hold many things

If you write a function that needs to return many objects, the only solution is
to place them inside a list. For example, consider this function:

```{r}
sqrt_newton <- function(a,
                        init = 1,
                        eps = 0.01,
                        steps = 1){

    stopifnot(a >= 0)
    while(abs(init**2 - a) > eps){
        init <- 1/2 *(init + a/init)
        steps <- steps + 1
    }
    list(
      "result" = init,
      "steps" = steps
    )
}

```

This function returns the square root of a number using Newton's algorithm, as
well as the number of steps, or iterations, it took to reach the solution:

```{r}
result_list <- sqrt_newton(1600)

result_list
```

It is quite common to print the number of steps to the console instead of
returning them. But the issue with a function that prints something to the
console instead of returning it, is that such a function is not pure, as it
changes something outside of its scope (it prints to the console!). And if you
need the information that got printed (for example, if you want to count all the
steps it took to run the script from start to finish), it is lost. It gets
printed, and that’s it. It is preferable to instead make the function pure by
returning everything inside a neat list. It is then possible to separately save
these objects if needed:

```{r}
result <- result_list$result

result_steps <- result_list$steps
```

Or you could define functions that know how to deal with the list:

```{r}

f <- function(result_list){
  list(
    "result" = result_list$result * 10,
    "steps" = result_list$steps + 1
    )
}

f(result_list)
```

It all depends on what you want to do. But it is usually better to keep
everything neatly inside a list.

Lists can also hold objects of different types:

```{r}
list(
  "a" = head(mtcars),
  "b" = ~lm(y ~ x)
  )
```

The list above has two elements, the first is the head of the `mtcars` data
frame, the second is a formula object. Lists can even hold other lists:

```{r}
list(
  "a" = head(mtcars),
  "b" = list(
    "c" = sqrt,
    "d" = my_plot # Remember this ggplot object from before?
    )
  )
```

Use this to your advantage.

### Lists as the cure to loops

Loops are incredibly useful, and you are likely familiar with them. The problem
with loops is that they are a concept from iterative programming, not functional
programming, and this is a problem because loops rely on changing the state of
your program to run. For example, let's suppose that you wish to use a for-loop
to compute the sum of the first 100 integers:

```{r}
result <- 0

for (i in 1:100){
  result <- result + i
}

print(result)
```

If you run `ls()` now, you should see that there's a variable `i` in your global
environment. This could cause issues further down in your pipeline if you need
to re-use `i`. Also, writing loops is, in my opinion, quite error prone. But how
can we avoid using loops? Looping in a functional programming language involves
using higher-order functions and lists. A reminder: a higher-order function is a
function that takes another function as an argument. Looping is a task like any
other, so I can write a function that does the looping. This function, which
I’ll call `looping()`, will take a function as an argument, as well as a
list. The list will serve as the container to hold our numbers:

```{r}
looping <- function(a_list, a_func, init = NULL, ...){

  # If the user does not provide an `init` value,
  # set the head of the list as the initial value
  if(is.null(init)){
    init <- a_list[[1]]
    a_list <- tail(a_list, -1)
  }

  # Separate the head from the tail of the list
  # and apply the function to the initial value and the head of the list
  head_list = a_list[[1]]
  tail_list = tail(a_list, -1)
  init = a_func(init, head_list, ...)

  # Check if we're done: if there is still some tail,
  # rerun the whole thing until there's no tail left
  if(length(tail_list) != 0){
    looping(tail_list, a_func, init, ...)
  }
  else {
    init
  }
}

```

Now, this might seem much more complicated than a for loop. However, now that we
have abstracted the loop away inside a function, we can keep reusing this
function:

```{r}
looping(as.list(seq(1, 100)), `+`)
```

Of course, because this is so useful, `looping()` actually ships with R, and is
called `Reduce()`:

```{r}
Reduce(`+`, seq(1, 100)) # the order of the arguments is `function` then `list` for `Reduce()`
```

But this is not the only way that we can loop. We can also write a loop that
applies a function to each element of a list, instead of operating on the whole
list:

```{r}
result <- as.list(seq(1, 5))
for (i in seq_along(result)){
  result[[i]] <- sqrt(result[[i]])
}

print(result)
```

Here again, we have to pollute the global environment by first creating a vessel
for our results, and then apply the function at each index. We can abstract this
process away in a function:

```{r}
applying <- function(a_list, a_func, ...){

  head_list = a_list[[1]]
  tail_list = tail(a_list, -1)
  result = a_func(head_list, ...)

  # Check if we're done: if there is still some tail, rerun the whole thing until there's no tail left
  if(length(tail_list) != 0){
    append(result, applying(tail_list, a_func, ...))
  }
  else {
    result
  }
}
```

Once again this might seem complicated, and I would agree. Abstraction is
complex. But once we have it, we can focus on the task at hand, instead of
having to always tell the computer what we want:

```{r}
applying(as.list(seq(1, 5)), sqrt)
```

Of course, R ships with its own, much more efficient, implementation of this
function:

```{r}
lapply(list(seq(1, 5)), sqrt)
```

In other programming languages, `lapply()` is often called `map()`. The
`{purrr}` package ships with other such useful higher-order functions that
abstract loops away. For example, there’s the function called `map2()`, that
maps a function of two arguments to each element of two atomic vectors or lists,
two at a time:

```{r}
library(purrr)

map2(
  .x = seq(1, 5),
  .y = seq(1, 5),
  .f = `+`
  )
```

If you have more than two lists, you can use `pmap()` instead.

Another important, idiomatic, way to deal with loops in R is to use matrix algebra
instead. For example, to compute the sum of the first 100 integers, the following
approach is possible:

```{r}
rep(1, 100) %*% seq(1, 100)
```

Also, don’t forget that many functions are vectorized by default, so no loop is required:

```{r}
sqrt(seq(1, 5))
```

or:

```{r}
seq(1, 5) + seq(1, 5)
```

Before diving directly into loops, check if the functions you’re using are
vectorized, or if there is a simple way to express the computation you want to
run in terms of matrix multiplication.

### Data frames

As mentioned in the introduction of this section, data frames are a special type
of list of atomic vectors. This means that just as I can use `lapply()` to
compute the square root of the elements of an atomic vector, as shown
previously, I can also operate on all the columns of a data frame. For example,
it is possible to determine the class of every column of a data frame like this:

```{r}
lapply(iris, class)
```

Unlike a list however, the elements of a data frame must be of the same length.
Data frames remain very flexible though, and using what we have learned until
now it is possible to use the data frame as a structure for all our
computations. For example, suppose that we have a data frame that contains data
on unemployment for the different subnational divisions of the Grand-Duchy of
Luxembourg, the country the author of this book hails from. Let’s suppose that I
want to generate several plots, per subnational division and per year.
Typically, we would use a loop for this, but we can use what we’ve learned here,
as well as some functions from the `{dplyr}`, `{purrr}`, `{ggplot2}` and
`{tidyr}` packages. I will be downloading data that I made available inside a
package, but instead of installing the package, I will download the `.rda` file
directly (which is the file format of packaged data) and then load that data
into our R session (instead of downloading from the long Github url, I download
the data from a shortened *is.gd* link):

```{r}
# Create a temporary file
unemp_path <- tempfile(fileext = ".rda")

# Download the data and save it to the path of the temporary file
# avoids having to install the package from Github
download.file(
  "https://is.gd/l57cNX",
  destfile = unemp_path)

# Load the data. The data is now available as 'unemp'
load(unemp_path)

```

Let’s load the required packages and take a look at the data:

```{r}
library(dplyr)
library(purrr)
library(ggplot2)
library(tidyr)

glimpse(unemp)

```

Column names are self-descriptive, but the `level` column needs some
explanations. `level` contains the administrative divisions of the country, so
the country of Luxembourg, then the Cantons and then the Communes.

Remember that Luxembourg can refer to the country, the canton or the commune of
Luxembourg. Now let’s suppose that I want a separate plot for the three communes
of Luxembourg, Esch-sur-Alzette and Wiltz. Instead of creating three separate
data frames and feeding them to the same ggplot code, I can instead take
advantage of the fact that data frames are lists, and are thus quite flexible.
Let’s start with filtering:

```{r}
filtered_unemp <- unemp %>%
  filter(
    level == "Commune",
    place_name %in% c("Luxembourg", "Esch-sur-Alzette", "Wiltz")
   )

glimpse(filtered_unemp)
```

We are now going to use the fact that data frames are lists, and that lists can
hold any type of object. For example, remember this list from before where one
of the elements is a data frame, and the second one a formula:

```{r}
list(
  "a" = head(mtcars),
  "b" = ~lm(y ~ x)
  )
```

`{dplyr}` comes with a function called `group_nest()` which groups the data
frame by a variable (such that the next computations will be performed
group-wise) and then nests the other columns into a smaller data frame. Let’s
try it and see what happens:


```{r}
nested_unemp <- filtered_unemp %>%
  group_nest(place_name)
```

Let’s see what this looks like:

```{r}
nested_unemp
```

`nested_unemp` is a new data frame of 3 rows, one per commune
("Esch-sur-Alzette", "Luxembourg", "Wiltz"), and of two columns, one for the
names of the communes, and the other contains every other variable inside a
smaller data frame. So this is a data frame that has one column where each
element of that column is itself a data frame. Such a column is called a
list-column. This is essentially a list of lists.

Let’s now think about this for a moment. If the column titled `data` is a list
of data frames, it should be possible to use a function like `map()` or
`lapply()` to apply a function on each of these data frames. Remember that
`map()` or `lapply()` require a list of elements of whatever type and a function
that accepts objects of this type as input. So this means that we could apply a
function that plots the data to each element of the column titled `data`. Since
each element of this column is a data frame, this function needs a data frame as
an input. As a first and simple example to illustrate this, let’s suppose that we
want to determine the number of rows of each data frame. This is how we would do
it:

```{r}
nested_unemp %>%
  mutate(nrows = map(data, nrow))
 # ’data’ is the name of
 # the list-column that contains
 # the smaller data frames
```

The new column, titled `nrows` is a list of integers. We can simplify it by
converting it directly to an atomic vector of integers by using `map_int()`
instead of `map()`:

```{r}
nested_unemp %>%
  mutate(nrows = map_int(data, nrow))
```

Let’s try a more complex example now. What if we want to filter rows (of course,
the simplest way would be to filter the rows we need before nesting the
data frame)? We need to apply the function `filter()` where its first argument
is a data frame and the second argument is a predicate:

```{r}
nested_unemp %>%
  mutate(nrows = map(data, \(x)filter(x, year == 2015)))
```

In this case, we need to use an anonymous function. This is because `filter()`
has two arguments and we need to make clear what it is we are mapping over and
what argument stays fixed; we are mapping over (iterating) the data frames but
the predicate `year == 2015` stays fixed.

We are now ready to plot our data. The best way to continue is to first get the
function right by creating one plot for one single commune. Let’s select the
dataset for the commune of `Luxembourg`:

```{r}
lux_data <- nested_unemp %>%
  filter(place_name == "Luxembourg") %>%
  unnest(data)
```

To plot this data, we can now write the required `ggplot2()` code:

```{r}
ggplot(data = lux_data) +
  theme_minimal() +
  geom_line(
    aes(year, unemployment_rate_in_percent, group = 1)
   ) +
  labs(title = "Unemployment in Luxembourg")

```

To turn the lines of code above into a function, you need to think about how
many arguments that function would have. There is an obvious one, the data
itself (in the snippet above, the data is the `lux_data` object). Another one
that is less obvious is in the title:

```
labs(title = "Unemployment in Luxembourg")
```

Ideally, we would want that title to change depending on the data set. So we
could write the function like so:

```{r}
make_plot <- function(x, y){
  ggplot(data = x) +
    theme_minimal() +
    geom_line(
      aes(year, unemployment_rate_in_percent, group = 1)
      ) +
    labs(title = paste("Unemployment in", y))
}

```

Let’s try it on our data:

```{r}
make_plot(lux_data, "Luxembourg")
```

Ok, so now, we simply need to apply this function to our nested data frame:

```{r}
nested_unemp <- nested_unemp %>%
  mutate(plots = map2(
    .x = data, # column of data frames
    .y = place_name, # column of commune names
    .f = make_plot
  ))

nested_unemp
```

If you look at the `plots` column, you see that it is a list of `gg` objects:
these are our plots. Let’s take a look at them:

```{r}
nested_unemp$plots
```

We could also have used an anonymous function (but it is more difficult to get
right):

```{r}
nested_unemp %>%
  mutate(plots2 = map2(
    .x = data,
    .y = place_name,
    .f = \(.x,.y)(
                ggplot(data = .x) +
                  theme_minimal() +
                  geom_line(
                    aes(year, unemployment_rate_in_percent, group = 1)
                   ) +
                  labs(title = paste("Unemployment in", .y))
                  )
           )
         ) %>%
  pull(plots2)

```

This list-column based workflow is extremely powerful and I highly advise you to
take the required time to master it. Remember, we never want to have to repeat
ourselves. This approach might seem more complicated than calling `make_plot()`
three times, but imagine that you need to do this for several countries, several
variables, etc... What are you going to do, copy and paste code everywhere? This
gets very tedious and more importantly, very error-prone, because now you’ve
just introduced many points of failure by having so much copy-pasted code. You
could of course use a loop instead of this list-column based workflow. But as
mentioned, the issue with loops is that you have to interact with the global
environment, which can lead to other issues. But whatever you end up using, you
need to avoid copy and pasting at all costs.

## Functional programming in R

Up until now I focused on general concepts rather than on specifics of the R
programming language when it comes to functional programming. In this section, we
will be focusing entirely on R-specific capabilities and packages for functional
programming.

### Base capabilities

R is a functional programming language (but not only), and as such it comes
with many functions out of the box to write functional code. We have already
discussed `lapply()` and `Reduce()`. You should know that depending on what you
want to achieve, there are other functions that are similar to `lapply()`:
`apply()`, `sapply()`, `vapply()`, `mapply()` and `tapply()`. There’s also
`Map()` which is a wrapper around `mapply()`. Each function performs the same
basic task of applying a function over all the elements of a list or list-like
structure, but it can be hard to keep them apart and when you should use one
over another. This is why `{purrr}`, which we will discuss in the next section,
is quite an interesting alternative to base R’s offering.

Another one of the quintessential functional programming functions (alongside
`Reduce()` and `Map()`) that ships with R is `Filter()`. If you know
`dplyr::filter()` you should be familiar with the concept of filtering rows of a
data frame where the elements of one particular column satisfy a predicate.
`Filter()` works the same way, but focusing on lists instead of data frame:

```{r}
Filter(is.character,
       list(
         seq(1, 5),
         "Hey")
       )
```

The call above only returns the elements where `is.character()` evaluates to
`TRUE`.

Another useful function is `Negate()` which is a function factory that takes a
boolean function as an input and returns the opposite boolean function. As an
illustration, suppose that in the example above we wanted to get everything
*but* the character:

```{r}
Filter(Negate(is.character),
       list(
         seq(1, 5),
         "Hey")
       )
```

There are some other functions like this that you might want to check out: type
`?Negate` in console to read more about them.

Sometimes you may need to run code with side-effects, but want to avoid any
interaction between these side-effects and the global environment. For example,
you might want to run some code that creates a plot and saves it to disk, or code
that creates some data and writes them to disk. `local()` can be used for this.
`local()` runs code in a temporary environment that gets discarded at the end:

```{r}
local({
  a <- 2
})
```

Variable `a` was created inside this local environment. Checking if it exists now
yields `FALSE`:

```{r}
exists("a")
```

We will be using this technique later in the book to keep our scripts pure.

Before continuing with R packages that extend R’s functional programming
capabilities it’s also important to stress that just as R is a functional
programming language, it is also an object oriented language. In fact, R is what
John Chambers called a *functional OOP* language (@chambers2014). I won’t delve
too much into what this means (read @wickham2019 for this), but as a short
discussion, consider the `print()` function. Depending on what type of object
the user gives it, it seems as if somehow `print()` knows what to do with it:

```{r}
print(5)
print(head(mtcars))
print(str(mtcars))
```

This works by essentially mixing both functional and object-oriented
programming, hence functional OOP. Let’s take a closer look at the source code
of `print()` by simply typing `print` without brackets, into a console:

```{r}
print
```

Quite unexpectedly, the source code of `print()` is one line long and is just
`UseMethod("print")`. So all `print()` does is use a generic method called
"print". If your text editor has auto-completion enabled, you might see that
there are actually many `print()` functions. For example, type
`print.data.frame` into a console:

```{r}
print.data.frame
```

This is the `print` function for `data.frame` objects. So what `print()` does,
is look at the class of its argument `x`, and then look for the right `print`
function to call. In more traditional OOP languages, users would type something
like:

```{r, eval = F}
mtcars.print()
```

In these languages, objects encapsulate methods (the equivalent of our
functions), so if `mtcars` is a data frame, it encapsulates a `print()` method
that then does the printing. R is different, because classes and methods are
kept separate. If a package developer creates a new object class, then the
developer also must implement the required methods. For example in the
`{chronicler}` package, the `chronicler` class is defined alongside the
`print.chronicler()` function to print these objects.

All of this to say that if you want to extend R by writing packages, learning
some OOP essentials is also important. But for data analysis, functional
programming does the job perfectly well. To learn more about R’s different OOP
systems (yes, R can do OOP in different ways and the one I sketched here is the
simplest, but probably the most used as well), take a look at @wickham2019.

### purrr

The `{purrr}` package, developed by Posit (formerly RStudio), contains many
functions to make functional programming with R more smooth. In the previous
section, we discussed the `apply()` family of function; they all do a very
similar thing, which is looping over a list and applying a function to the
elements of the list, but it is not quite easy to remember which one does what.
Also, for some of these functions like `apply()`, the list argument comes first,
and then the function, but in the case of `mapply()`, the function comes first.
This type of inconsistencies can be frustrating. Another issue with these
functions is that it is not always easy to know what type the output is going to
be. List? Atomic vector? Something else?

`{purrr}` solves this issue by offering the `map()` family of functions, which
behave in a very consistent way. The basic function is called `map()` and we’ve
already used it:

```{r}
map(seq(1, 5), sqrt)
```

But there are many interesting variants:

```{r}
map_dbl(seq(1, 5), sqrt)
```

`map_dbl()` coerces the output to an atomic vector of doubles instead of a list
of doubles. Then there’s:

```{r}
map_chr(letters, toupper)
```

for when the output needs to be an atomic vector of characters.

There are many others, so take a look at the documentation with `?map`. There’s
also `walk()` which is used if you’re only interested in the side-effect of the
function (for example if the function takes paths as input and saves something
to disk).

`{purrr}` also has functions to replace `Reduce()`, simply called `reduce()` and
`accumulate()`, and there are many, many other useful functions. Read through
the [documentation of the
package](https://purrr.tidyverse.org/reference/index.html)^[https://purrr.tidyverse.org/reference/index.html]
and take the time to learn about all it has to offer.

### withr

`{withr}` is a powerful package that makes it easy to "purify" functions that
behave in a way that can cause problems. Remember the function from the
introduction that randomly gave out a dish Bruno liked? Here it is again:

```{r}
h <- function(name, food_list = list()){

  food <- sample(c("lasagna", "cassoulet", "feijoada"), 1)

  food_list <- append(food_list, food)

  print(paste0(name, " likes ", food))

  food_list
}
```

For the same input, this function may return different outputs so this function
is not referentially transparent. So we improved the function by adding calls to
`set.seed()` like this:

```{r}
h2 <- function(name, food_list = list(), seed = 123){

  # We set the seed, making sure that we get the same selection of food for a given seed
  set.seed(seed)
  food <- sample(c("lasagna", "cassoulet", "feijoada"), 1)

  # We now need to unset the seed, because if we don't, guess what, the seed will stay set for the whole session!
  set.seed(NULL)

  food_list <- append(food_list, food)

  print(paste0(name, " likes ", food))

  food_list
}
```

The problem with this approach is that we need to modify our function. We can
instead use `withr::with_seed()` to achieve the same effect:

```{r}
withr::with_seed(seed = 123,
                 h("Bruno"))
```

It is also easier to create a wrapper if needed:

```{r}
h3 <- function(..., seed){
  withr::with_seed(seed = seed,
                   h(...))
}
```

```{r}
h3("Bruno", seed = 123)
```

In a previous example we downloaded a dataset and loaded it into memory; we did
so by first creating a temporary file, then downloading it and then loading it.
Suppose that instead of loading this data into our session, we simply wanted to
test whether the link was still working. We wouldn’t want to keep the loaded
data in our session, so to avoid having to delete it again manually, we could
use `with_tempfile()`:

```{r, warning = FALSE}
withr::with_tempfile("unemp", {
  download.file(
    "https://is.gd/l57cNX",
    destfile = unemp)
  load(unemp)
  nrow(unemp)
  }
)

```

The data got downloaded, and then loaded, and then we computed the number of
rows of the data, without touching the global environment, or state, of our
current session.

Just like for `{purrr}`, `{withr}` has many useful functions which I encourage
you to [familiarize yourself
with](https://withr.r-lib.org/reference/index.html)^[https://withr.r-lib.org/reference/index.html].

## Conclusion

If there is only one thing that you should remember from this chapter, it would
be pure functions. Writing pure functions is in my opinion not very difficult to
do and comes with many benefits. But avoiding loops and replacing them with
higher-order functions (`lapply()`, `Reduce()`, `purrr::map()` -- and its
variants --) also pays off. While this chapter stresses the advantages of
functional programming, you should not forget that R is not a pure, and solely,
functional programming language and that other paradigms, like object-oriented
programming, are also available to you. So if your goal is to master the
language (instead of "just" using it to solve data analysis problems), then you
also need to know about R’s OOP capabilities.