bookclub-advr

html.json (13253B)

---

 {
   "hash": "498fd72af1822effcb92021a43a9d7c2",
   "result": {
     "engine": "knitr",
     "markdown": "---\nengine: knitr\ntitle: Measuring performance\n---\n\n## Learning objectives:\n\n- Understand how to improve your code for making it faster\n- Learn what are the tools for improving your code\n- Test how to profile your code\n\n\n## Introduction\n\n> \"Before you can make your code faster, you first need to figure out what’s making it slow.\"\n\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n![SLOW DOWN TO LEARN HOW TO CODE FASTER | credits: packtpub.com](images/23_code_faster.jpeg){fig-align='center' width=348}\n:::\n:::\n\n\n\n- **profile** your code: measure the run-time of each line of code using realistic inputs\n- **experiment** with alternatives to find faster code\n- **microbenchmark** to measure the difference in performance.\n\n\n\n## Profiling\n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(profvis)\nlibrary(bench)\n```\n:::\n\n\n\nThe tool to use is a **profiler**, it allows for **sampling** the code performance through stopping the execution of code every few milliseconds and recording all the steps.\n\nExample:\n\n\n::: {.cell}\n\n```{.r .cell-code}\nf <- function() {\n  pause(0.1)\n  g()\n  h()\n}\ng <- function() {\n  pause(0.1)\n  h()\n}\nh <- function() {\n  pause(0.1)\n}\n```\n:::\n\n\nProfile the execution of f():\n\n    profvis::pause() is used instead of Sys.sleep()\n    profile f(), with utils::Rprof()\n    \n\n::: {.cell}\n\n```{.r .cell-code}\ntmp <- tempfile()\nRprof(tmp, interval = 0.1)\nf()\n```\n\n::: {.cell-output .cell-output-stdout}\n\n```\n#> NULL\n```\n\n\n:::\n\n```{.r .cell-code}\nRprof(NULL)\nwriteLines(readLines(tmp))\n```\n\n::: {.cell-output .cell-output-stdout}\n\n```\n#> sample.interval=100000\n#> \"pause\" \"f\" \"eval\" \"eval\" \"withVisible\" \"withCallingHandlers\" \"eval\" \"eval\" \"with_handlers\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"withRestarts\" \"evaluate::evaluate\" \"evaluate\" \"in_dir\" \"in_input_dir\" \"eng_r\" \"block_exec\" \"call_block\" \"process_group\" \"withCallingHandlers\" \"xfun:::handle_error\" \"process_file\" \"knitr::knit\" \"rmarkdown::render\" \"execute\" \".main\" \n#> \"pause\" \"g\" \"f\" \"eval\" \"eval\" \"withVisible\" \"withCallingHandlers\" \"eval\" \"eval\" \"with_handlers\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"withRestarts\" \"evaluate::evaluate\" \"evaluate\" \"in_dir\" \"in_input_dir\" \"eng_r\" \"block_exec\" \"call_block\" \"process_group\" \"withCallingHandlers\" \"xfun:::handle_error\" \"process_file\" \"knitr::knit\" \"rmarkdown::render\" \"execute\" \".main\" \n#> \"pause\" \"h\" \"g\" \"f\" \"eval\" \"eval\" \"withVisible\" \"withCallingHandlers\" \"eval\" \"eval\" \"with_handlers\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"withRestarts\" \"evaluate::evaluate\" \"evaluate\" \"in_dir\" \"in_input_dir\" \"eng_r\" \"block_exec\" \"call_block\" \"process_group\" \"withCallingHandlers\" \"xfun:::handle_error\" \"process_file\" \"knitr::knit\" \"rmarkdown::render\" \"execute\" \".main\" \n#> \"pause\" \"h\" \"f\" \"eval\" \"eval\" \"withVisible\" \"withCallingHandlers\" \"eval\" \"eval\" \"with_handlers\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"doWithOneRestart\" \"withOneRestart\" \"withRestartList\" \"withRestarts\" \"evaluate::evaluate\" \"evaluate\" \"in_dir\" \"in_input_dir\" \"eng_r\" \"block_exec\" \"call_block\" \"process_group\" \"withCallingHandlers\" \"xfun:::handle_error\" \"process_file\" \"knitr::knit\" \"rmarkdown::render\" \"execute\" \".main\"\n```\n\n\n:::\n:::\n\n    \n    \n**Visualising profiles**\n\nMakes easier to build up a mental model of what you need to change:\n\n    profvis::profvis()\n    utils::summaryRprof()\n\n\n::: {.cell}\n\n```{.r .cell-code}\nsource(\"scripts/profiling-example.R\")\nprofvis(f())\n```\n\n::: {.cell-output-display}\n\n```{=html}\n<div class=\"profvis html-widget html-fill-item\" id=\"htmlwidget-a76651cfc65320a86296\" style=\"width:100%;height:600px;\"></div>\n<script type=\"application/json\" data-for=\"htmlwidget-a76651cfc65320a86296\">{\"x\":{\"message\":{\"prof\":{\"time\":[1,1,2,2,3,3,4,4,5,5,6,6,7,7,7,8,8,8,9,9,9,10,10,10,11,11,11,12,12,12,13,13,13,14,14,14,14,15,15,15,15,16,16,16,16,17,17,17,17,18,18,18,18,19,19,19,19,20,20,20,21,21,21,22,22,22,23,23,23,24,24,24,25,25,25],\"depth\":[2,1,2,1,2,1,2,1,2,1,2,1,3,2,1,3,2,1,3,2,1,3,2,1,3,2,1,3,2,1,3,2,1,4,3,2,1,4,3,2,1,4,3,2,1,4,3,2,1,4,3,2,1,4,3,2,1,3,2,1,3,2,1,3,2,1,3,2,1,3,2,1,3,2,1],\"label\":[\"pause\",\"f\",\"pause\",\"f\",\"pause\",\"f\",\"pause\",\"f\",\"pause\",\"f\",\"pause\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"g\",\"f\",\"pause\",\"h\",\"g\",\"f\",\"pause\",\"h\",\"g\",\"f\",\"pause\",\"h\",\"g\",\"f\",\"pause\",\"h\",\"g\",\"f\",\"pause\",\"h\",\"g\",\"f\",\"pause\",\"h\",\"g\",\"f\",\"pause\",\"h\",\"f\",\"pause\",\"h\",\"f\",\"pause\",\"h\",\"f\",\"pause\",\"h\",\"f\",\"pause\",\"h\",\"f\",\"pause\",\"h\",\"f\"],\"filenum\":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],\"linenum\":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],\"memalloc\":[10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281,10.16792297363281],\"meminc\":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],\"filename\":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null]},\"interval\":10,\"files\":[],\"prof_output\":\"C:\\\\Users\\\\jonth\\\\AppData\\\\Local\\\\Temp\\\\RtmpgFLX7B\\\\file8a2412f12f4c.prof\",\"highlight\":{\"output\":[\"^output\\\\$\"],\"gc\":[\"^<GC>$\"],\"stacktrace\":[\"^\\\\.\\\\.stacktraceo(n|ff)\\\\.\\\\.$\"]},\"split\":\"h\"}},\"evals\":[],\"jsHooks\":[]}</script>\n```\n\n:::\n:::\n\n\n**Memory profiling and the garbage collector**\n\nProfiling a loop that modifies an existing variable:\n\n::: {.cell}\n\n```{.r .cell-code}\nprofvis::profvis({\n  x <- integer()\nfor (i in 1:1e4) {\n  x <- c(x, i)\n}\n})\n```\n\n::: {.cell-output-display}\n\n```{=html}\n<div class=\"profvis html-widget html-fill-item\" id=\"htmlwidget-a23be7babeaed78bc39a\" style=\"width:100%;height:600px;\"></div>\n<script type=\"application/json\" data-for=\"htmlwidget-a23be7babeaed78bc39a\">{\"x\":{\"message\":{\"prof\":{\"time\":[1,1,1,2,2,3,3,4,4,5,5,6,6,6],\"depth\":[3,2,1,2,1,2,1,2,1,2,1,3,2,1],\"label\":[\"Rprof\",\"profvis::profvis\",\".main\",\"c\",\".main\",\"c\",\".main\",\"c\",\".main\",\"c\",\".main\",\"<GC>\",\"c\",\".main\"],\"filenum\":[null,null,null,null,null,null,null,null,null,null,null,null,null,null],\"linenum\":[null,null,null,null,null,null,null,null,null,null,null,null,null,null],\"memalloc\":[10.79654693603516,10.79654693603516,10.79654693603516,45.85549163818359,45.85549163818359,27.03416442871094,27.03416442871094,54.31883239746094,54.31883239746094,30.6883544921875,30.6883544921875,36.67101287841797,36.67101287841797,36.67101287841797],\"meminc\":[0,0,0,35.05894470214844,0,-18.82132720947266,0,27.28466796875,0,-23.63047790527344,0,5.982658386230469,0,0],\"filename\":[null,null,null,null,null,null,null,null,null,null,null,null,null,null]},\"interval\":10,\"files\":[],\"prof_output\":\"C:\\\\Users\\\\jonth\\\\AppData\\\\Local\\\\Temp\\\\RtmpgFLX7B\\\\file8a24629a5d4c.prof\",\"highlight\":{\"output\":[\"^output\\\\$\"],\"gc\":[\"^<GC>$\"],\"stacktrace\":[\"^\\\\.\\\\.stacktraceo(n|ff)\\\\.\\\\.$\"]},\"split\":\"h\"}},\"evals\":[],\"jsHooks\":[]}</script>\n```\n\n:::\n:::\n\n\nYou can figure out what is the source of the problem by looking at the memory column. In this case, **copy-on-modify** acts in each iteration of the loop creating another copy of x.\n\n\n**Limitations**\n\n- Profiling does not extend to C code\n- Anonymous functions are hard to figure out\n- Arguments are evaluated inside another function\n\n\n### Exercise\n\n::: {.cell}\n\n```{.r .cell-code}\nprofvis::profvis({\n  f <- function(n = 1e5) {\n  x <- rep(1, n)\n  rm(x)\n}\n},torture = TRUE)\n```\n:::\n\n\n    ?rm()\n    \n[solution](https://advanced-r-solutions.rbind.io/measuring-performance.html)    \n    \n## Microbenchmarking\n\n\n*Measurement of the performance of a very small piece of code* is useful for comparing small snippets of code for specific tasks.\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n![Credits: Google search-engine](images/23_microbenchmarking.jpeg){fig-align='center' width=159}\n:::\n:::\n\n\n\nThe {bench} package uses a high precision time.\n\n    bench::mark()\n    \n    \n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(bench)\nx <- runif(100)\n(lb <- bench::mark(\n  sqrt(x),\n  x ^ 0.5\n))\n```\n\n::: {.cell-output .cell-output-stdout}\n\n```\n#> # A tibble: 2 × 6\n#>   expression      min   median `itr/sec` mem_alloc `gc/sec`\n#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>\n#> 1 sqrt(x)       300ns    400ns  1977692.      848B        0\n#> 2 x^0.5         2.9µs      3µs   321570.      848B        0\n```\n\n\n:::\n:::\n\n- heavily right-skewed distribution\n\n\n\n::: {.cell}\n\n```{.r .cell-code}\nrequire(ggbeeswarm)\n```\n\n::: {.cell-output .cell-output-stderr}\n\n```\n#> Loading required package: ggbeeswarm\n```\n\n\n:::\n\n::: {.cell-output .cell-output-stderr}\n\n```\n#> Loading required package: ggplot2\n```\n\n\n:::\n\n```{.r .cell-code}\nplot(lb)\n```\n\n::: {.cell-output-display}\n![](23_files/figure-html/unnamed-chunk-10-1.png){width=672}\n:::\n:::\n\n\n\n## Resources\n\n- [profvis package](https://rstudio.github.io/profvis/)\n- [bench package](https://cran.r-project.org/web/packages/bench/bench.pdf)\n- [solutions](https://advanced-r-solutions.rbind.io/measuring-performance.html)\n",
     "supporting": [
       "23_files"
     ],
     "filters": [
       "rmarkdown/pagebreak.lua"
     ],
     "includes": {
       "include-in-header": [
         "<link href=\"../site_libs/htmltools-fill-0.5.8.1/fill.css\" rel=\"stylesheet\" />\n<script src=\"../site_libs/htmlwidgets-1.6.4/htmlwidgets.js\"></script>\n<script src=\"../site_libs/jquery-3.7.1/jquery.min.js\"></script>\n<script src=\"../site_libs/d3-3.5.6/d3.min.js\"></script>\n<link href=\"../site_libs/profvis-0.3.6.9000/profvis.css\" rel=\"stylesheet\" />\n<script src=\"../site_libs/profvis-0.3.6.9000/profvis.js\"></script>\n<script src=\"../site_libs/profvis-0.3.6.9000/scroll.js\"></script>\n<link href=\"../site_libs/highlight-11.10.0/textmate.css\" rel=\"stylesheet\" />\n<script src=\"../site_libs/highlight-11.10.0/highlight.min.js\"></script>\n<script src=\"../site_libs/profvis-binding-0.4.0/profvis.js\"></script>\n"
       ]
     },
     "engineDependencies": {},
     "preserve": {},
     "postProcess": true
   }
 }