```{r setup, include=FALSE}

The biggest difference between R and Python is not where R starts counting,

but its use of [lazy evaluation](glossary.html#lazy-evaluation).

Nothing in R truly makes sense until we understand how this works.

- Trace the order of evaluation in function calls.

- Explain what environments and expressions are and how they relate to one another.

- Justify the author's use of ASCII art in the second decade of the 21st Century.

Let's start by looking at a small Python program and its output:

```{python py-def-call}

When we call `tens_func` we pass it `initial + " more"`;

since `initial` has just been assigned the value `"start"`,

that's the same as calling `tens_func` with `"start more"`.

`tens_func` then calls `ones_func` with `"start more tens"`,

and `ones_func` returns `"start more tens ones"`.

But there's more going on here than that two-sentence summary suggests.

Let's spell out the steps:

```{python py-def-call-details}

Step 1: we assign `"start"` to `initial` at the global level:

```{r py-step-1, echo=FALSE, fig.cap="Python Step 1"}

Step 2: we ask Python to call `tens_func(initial + "more")`,

so it creates a temporary variable to hold the result of the concatenation

*before* calling `tens_func`:

```{r py-step-2, echo=FALSE, fig.cap="Python Step 2"}

Step 3: Python creates a new stack frame to hold the call to `tens_func`:

```{r py-step-3, echo=FALSE, fig.cap="Python Step 3"}

Note that `tens_arg` points to the same thing in memory as `global_temp_1`,

since Python passes everything by reference.

Step 4: we ask Python to call `ones_func(tens_arg + " tens")`,

so it creates another temporary variable:

```{r py-step-4, echo=FALSE, fig.cap="Python Step 4"}

Step 5: Python creates a new stack frame to manage the call to `ones_func`:

```{r py-step-5, echo=FALSE, fig.cap="Python Step 5"}

Step 6: Python creates a temporary variable to hold `ones_arg + "ones"`:

```{r py-step-6, echo=FALSE, fig.cap="Python Step 6"}

Step 7: Python returns from `ones_func`

and puts its result in yet another temporary variable in `tens_func`:

```{r py-step-7, echo=FALSE, fig.cap="Python Step 7"}

Step 8: Python returns from `tens_func`

and puts that call's result in `final`:

```{r py-step-8, echo=FALSE, fig.cap="Python Step 8"}

The most important thing here is that Python evaluates expressions *before* it calls functions,

and passes the results of those evaluations to the functions.

This is called [eager evaluation](glossary.html#eager-evaluation),

and is what most widely-used programming languages do.

In contrast,

R uses [lazy evaluation](glossary.html#lazy-evaluation).

Here's an R program that's roughly equivalent to the Python shown above:

```{r r-def-call}

And here it is with the intermediate steps spelled out in a syntax I just made up:

```{r r-def-call-details, eval=FALSE}

While the original code looked much like our Python,

the evaluation trace is very different,

and hinges on the fact that

*an expression in a programming language can be represented as a data structure*.

Step 1: we assign "start" to `initial` in the [global environment](glossary.html#global-environment):

```{r r-step-1, echo=FALSE, fig.cap="R Step 1"}

Step 2: we ask R to call `tens_func(initial + "more")`,

so it creates a [promise](glossary.html#promise) to hold:

- the [environment](glossary.html#environment) we're in (which I'm surrounding with `@`),

- the expression we're passing to the function, and

- the value of that expression (which I'm showing as `____`, since it's initially empty).

```{r r-step-2, echo=FALSE, fig.cap="R Step 2"}

and in Step 3,

passes that into `tens_func`:

```{r r-step-3, echo=FALSE, fig.cap="R Step 3"}

Crucially,

the promise in `tens_func` remembers that it was created in the [global environment](glossary.html#global-environment):

it will eventually need a value for `initial`,

so it needs to know where to look to find the right one.

Step 4: since the very next thing we ask for is `paste(tens_arg, "tens")`,

R needs a value for `tens_arg`.

To get it,

R evaluates the promise that `tens_arg` refers to:

```{r r-step-4 ,echo=FALSE, fig.cap="R Step 4"}

This evaluation happens *after* `tens_func` has been called,

not before as in Python,

which is why this scheme is called "lazy" evaluation.

Once a promise has been resolved,

R uses its value,

and that value never changes.

Steps 5:

`tens_func` wants to call `ones_func`,

so R creates another promise to record what's being passed into `ones_func`:

```{r r-step-5, echo=FALSE, fig.cap="R Step 5"}

Step 6:

R calls `ones_func`,

binding the newly-created promise to `ones_arg` as it does so:

```{r r-step-6, echo=FALSE, fig.cap="R Step 6"}

Step 7:

R needs a value for `ones_arg` to pass to `paste`,

so it resolves the promise:

```{r r-step-7, echo=FALSE, fig.cap="R Step 7"}

Step 8: `ones_func` uses `paste` to concatenate strings:

```{r r-step-8, echo=FALSE, fig.cap="R Step 8"}

Step 9: `ones_func` returns:

```{r r-step-9, echo=FALSE, fig.cap="R Step 9"}

Step 10: `tens_func` returns:

```{r r-step-10, echo=FALSE, fig.cap="R Step 10"}

We got the same answer as we did in Python,

but in a significantly different way.

Each time we passed something into a function,

R created a promise to record what it was and where it came from,

and then resolved the promise when the value was needed.

R *always* does this—if we call:

```{r call-sign, eval=FALSE}

then behind the scenes,

R is creating a promise and passing it to `sign`,

where it is automatically resolved to get the number 2 when its value is needed.

(If I wanted to be thorough,

I would have shown the promises passed into `paste` at each stage of execution above.)

R's lazy evaluation seems pointless

if it always produces the same answer as Python's eager evaluation,

but it doesn't have to.

To see how powerful lazy evaluation can be,

let's create an expression of our own:

```{r create-expr}

Displaying the value of `my_expr` isn't very exciting:

```{r show-expr}

but what kind of thing is it?

```{r type-of-expr}

A symbol is a kind of expression.

It is not a string (though strings can be converted to symbols and symbols to strings)

nor is it a value—not yet.

If we try to get the value it refers to, R displays an error message:

```{r eval-expr-error, error=TRUE}

We haven't created a variable called `red`,

so R cannot evaluate an expression that asks for it.

But what if we create such a variable now and then re-evaluate the expression?

```{r eval-after-var-def}

More usefully,

what if we create something that has a value for `red`:

```{r create-tibble}

and then ask R to evaluate our expression in the [context](glossary.html#context) of that tibble:

```{r eval-with-context}

When we do this,

`eval` looks for definitions of variables in the data structure we've given it—in this case,

the tibble `color_data`.

Since that tibble has a column called `red`,

`eval(my_expr, color_data)` gives us that column.

This may not seem life-changing yet,

but being able to pass expressions around

and evaluate them in contexts of our choosing allows us to seem very clever indeed.

For example,

let's create another expression:

```{r expr-add-red-green}

The type of `add_red_green` is `language` rather than `symbol` because it contains more than just a single symbol,

but it's still an expression,

so we can evaluate it in the context of our data frame:

```{r eval-add-red-green}

Still not convinced?

Have a look at this function:

```{r def-run-many-checks}

`run_many_checks` takes a tibble and some logical expressions,

evaluates each expression in turn,

and returns a list of results:

```{r call-many-checks}

We can take it one step further and simply report whether all the checks passed or not:

```{r report-run-all-checks}

This is cool, but typing `expr(...)` over and over is kind of clumsy.

It also seems superfluous,

since we know that arguments aren't evaluated before they're passed into functions.

Can we get rid of this and write something that does this?

```{r desired-check-call, eval=FALSE}

The answer is going to be "yes",

but it's going to take a bit of work.

without calling `expr` to quote our expressions explicitly.

For simplicity's sake,

our first attempt only handles a single expression:

```{r def-check-naive}

When we try it, it fails:

```{r check-naive-fails, error=TRUE}

This actually makes sense:

by the time we reach the call to `eval`,

`test` refers to a promise that represents the value of `red != green` in the global environment.

Promises are not expressions—each promise contains an expression,

but it also contains an environment and a copy of the expression's value (if it has ever been calculated).

As a result,

when R sees the call to `eval` inside `check_naive`

it automatically tries to resolve the promise that contains `left != right`,

and fails because there are no variables with those names in the global environment.

So how can we get the expression out of the promise without triggering evaluation?

One way is to use a function called `substitute`:

```{r check-using-substitute}

However,

`substitute` is frowned upon because it does one thing when called interactively on the command line

and something else when called inside a function.

Instead,

we can use a function called `enquo` from the rlang package.

`enquo` returns an object called a [quosure](glossary.html#quosure)

that contains only an unevaluated expression and an environment:

```{r check-with-enquo-fail}

Ah: a quosure is a structured object,

so evaluating it just gives it back to us in the same way that evaluating `2` or `"hello"` would.

What we want to `eval` is the expression inside the quosure,

which we can get using `quo_get_expr`:

```{r check-with-enquo-success}

Enquoting and evaluating expressions is done so often in the tidyverse

that rlang provides a shortcut called `{{..}}`:

We can use this to write a function `run_two_checks` that runs two checks on some data:

That's much easier to follow than a bunch of `enquo` and `eval` calls,

but what if we want to handle an arbitrary number of checks?

Our first attempt is this:

```{r error=TRUE}

This code fails because the call to `list(...)` tries to evaluate the expressions in `...`

when adding them to the list.

What we need to use instead is `enquos`,

which does what `enquo` does but on `...`:

We will occasionally need to go one level deeper in tidy evaluation.

Our first attempt (which only handles a single test) is going to fail on purpose

to demonstrate a common mistake:

```{r deliberate-failure, error=TRUE}

That failed because we're not enquoting the test.

Let's modify it the code to enquote and then pass in the expression:

```{r another-failure, error=TRUE}

Damn—we thought this one had a chance.

The problem is that when we say `result = x_test`,

what actually gets passed into `transmute` is a promise containing an expression.

Somehow,

we need to prevent R from doing that promise wrapping.

This brings us to `enquo`'s partner `!!`,

which we can use to [splice](glossary.html#splice) the expression in a quosure into a function call.

`!!` is pronounced "bang bang" or "oh hell",

depending on how your day is going.

It only works in contexts like function calls where R is automatically quoting things for us,

but if we use it then,

it does exactly what we want:

```{r check-using-bang-bang, error=TRUE}

We are almost in a state of grace.

The two rules we must follow are:

1. Use `enquo` to enquote every argument that contains an unevaluated expression.

2. Use `!!` when passing each of those arguments into a tidyverse function.

```{r check-all-complete}

And just to make sure that it fails when it's supposed to:

```{r check-all-pass}

Backing up a bit,

`!!` works because there are

[two kinds of functions in R](https://tidyeval.tidyverse.org/getting-up-to-speed.html#whats-special-about-quoting-functions):

[evaluating functions](glossary.html#evaluating-function) and

[quoting functions](glossary.html#quoting-function).

Evaluating functions take arguments as values—they're what most of us are used to working with.

Quoting functions, on the other hand, aren't passed the values of expressions, but the expressions themselves.

When we write `color_data$red`, the `$` function is being passed `color_data` and the quoted expression `red`.

This is why we can't use variables as field names with `$`:

```{r cannot-use-variables-with-dollar, error=TRUE}

The square bracket operators `[` and `[[`, on the other hand,

are evaluating functions,

so we can give them a variable containing a column name and get either a single-column tibble:

```{r can-use-variables-with-brackets-1}

or a naked vector:

```{r can-use-variables-with-brackets-2}

Delayed evaluation and quoting are confusing for two reasons:

1. They expose machinery that most programmers have never had to deal with before (and might not even have known existed).

It's rather like learning to drive an automatic transmission and then switching to a manual one—all of a sudden

you have to worry about a gear shift and a clutch.

2. R's built-in tools don't behave as consistently as they could,

and the core functions provided by the tidverse as alternatives use variations on a small number of names:

`quo`, `quote`, and `enquo` might all appear on the same page.

That said,

being able to pass column names to functions without wrapping them in strings is very useful,

and many powerful tools (such as using formulas in models)

rely on taking unevaluated expressions apart and rearranging them.

If you would like to know more,

or check that what you now think you understand is accurate,

[this tutorial][lyttle-tutorial] is a good next step.

```{r keypoints, child="keypoints/nse.md"}

```{r links, child="etc/links.md"}