Performance

posit::conf(2023)
Shiny in Production: Tools & Techniques

Performance Agenda

• Profiling your Shiny app with {profvis}
• Lightning-quick data loading/querying with {arrow} & .parquet files
• Asynchronous processes with {crew}

{profvis}: an R package for profiling R code
(including Shiny apps)

What is {profvis}?

R package for visualizing how (and how fast/slow) your R code runs

{profvis} website

• The package is from Posit
• Help you understand the bottlenecks in your R code
• and, you guessed it, it works with Shiny too

Working with {profvis} & {golem}

Our dev/run_dev.R script is where we can profile our app interactively.

options(golem.app.prod = FALSE)

golem::detach_all_attached()

golem::document_and_reload()

run_app()

Working with {profvis} & {golem}

Our dev/run_dev.R script is where we can profile our app interactively.

options(golem.app.prod = FALSE)

golem::detach_all_attached()

golem::document_and_reload()

profvis::profvis({
  print(run_app())
})

You need to wrap the run_app() function in print(), before passing it to profvis::profvis()

Demo!

• Show how to change the dev/run_dev.R script to enable {profvis} profiling
• Launch the app, navigate to the “Overview” page, and change a filter
• Stop the app, and wait for the report to load
• Describe the flamegraph, change the filters to only show events that took time
• Navigate to the ‘data’ tab and discuss what took the most time

Your Turn: Exercise 1

Profile the LEGO Bricks app!

• Details
• Posit Cloud project: Performance Exercise 1

05:00

{arrow} & the .parquet file format

What are .parquet files?

• .parquet is a columnar storage format
• .parquet files not only store data, but they also store metadata about your data (i.e., data types for each column, number of rows in the file, etc.)
• Smaller files
• Faster read speed

• HOT TAKE INCOMING: parquet is the new csv
• parquet files typically are the data structure that lives behind projects like the open source Delta Lake
• faster across pretty much all benchmarks

What is the {arrow} R package?

• Part of the larger Apache Arrow project
• Connect to your data with {arrow}…
• … and query it with {dplyr}

Apache Arrow Homepage

Shiny + Arrow Article

• “multi-language toolbox for accelerated data interchange and in-memory processing”
• I.e., a set of data manipulation standards (particularly against parquet files) that has been implemented in a bunch of languages including R, Python, Rust, Go, and more
• {arrow} let’s you use {dplyr} verbs against a single parquet file (or, perhaps more importantly, a set of parquet files) to query the data in those files
• When it comes to building Shiny apps, we should look for easy places where we can gain efficiency & speed to improve our user experience (you don’t want users waiting 20 seconds for your data prep logic to run against a single massive csv); it’s very likely that the combination of .parquet + {arrow} + {dplyr} can meet your app performance needs (it does for at least 95% of my use cases – there are very few cases where I have to go beyond that and start looking into other engines for faster data manipulation)

Code-Along

Using .parquet in the LEGO Bricks Shiny app

• Details
• Posit Cloud project: Performance Exercise 1

Async Processing

Single (threaded) Line

• A single R process managing the different tasks in a Shiny application
• Executed one-by-one

Should I care? It Depends …

If you are the only user for a quick and efficient app: Likely not!

Crowd Pleaser

Multiple users accessing the app concurrently:

• Single-threaded R process serving multiple users in typical deployments

Asynchronous Processing (circa 2018)

📦 {promises}

Handle objects representing the (eventual) result of an async operation

📦 {future}

Launch tasks without blocking current R session

Introducing {crew}

A distributed worker launcher for asynchronous tasks

• Extends use of the mirai task scheduler to multiple computing backends
• Central controller object manages tasks (scales on fly)
• Supports multiple controller groups for specialized worker types
• Fits nicely with {targets} and …

Watch-Along

Using {crew} inside a Shiny application:

• Vignette: https://wlandau.github.io/crew/articles/shiny.html
• Application: https://wlandau.shinyapps.io/crew-shiny

Setting up for Success

1. Create functions for long-running tasks
2. Create multiple reactiveVal objects for bookkeeping
3. Set up a {crew} controller
4. Establish an event-driven push of task to the controller with monitoring of worker status

Code-Along

Asynchronous calls of a model prediction API.

• Details
• Posit Cloud project: Performance Code-along 2