RAP in Practice

Senior Statisticians Away Day

Alice Byers

Data Innovation Team, Data Division

22 November 2023

About me

  • Reproducible Analytical Pipeline (RAP) developer

  • Background in statistics and data analysis

Why should I care about RAP?


  • Less time spent producing routine analysis; more time for other work

  • Better quality analysis

What is RAP?

  • An overall approach to carrying out analysis

  • A set of principles that ensure analysis is:

    • reproducible
    • auditable
    • efficient
    • high quality

Examples of projects

  • Routine ‘traditional’ publications

  • Shiny dashboards

  • Ministerial briefings

  • FOIs

  • One-off analysis

Stages of analysis

  • Data extraction

  • Data cleaning

  • Data analysis

  • Modelling

  • Data visualisation

  • Reporting

RAP principles

In order to achieve the full benefits, at a minimum a RAP must:

  • Minimise manual steps

  • Be built using open-source software; e.g. R, Python

  • Be peer reviewed by colleagues

  • Be version controlled; e.g. Git

  • Be open to anyone; e.g. code published on GitHub

  • Follow good practice for quality assurance

  • Contain well-commented code and have documentation embedded

Where to start

  • Following all 7 principles will help you to achieve the full benefits

  • Following one or some of the principles will still bring benefits!

  • Start small, review and iterate

RAP Support

  • We can help with:

    • Reviewing project requirements and mapping out a process.

    • Deciding where to start when transitioning a process to a RAP.

    • Understanding what software is available and how to make best use of it.

    • Following best practices for structuring scripts, files and folders for your RAP.

    • Using Git and GitHub, ensuring your RAP project is setup within a repository.

    • Applying best practices to your code, for example “do not repeat yourself” and following coding conventions from a style guide.

    • Reviewing your code.

RAP Support

  • Get in touch to discuss your project

  • Long term support

    • The aim is to help you to develop your RAP skills, not to write all the code for you.

    • Agreement to work together for 4-6 months

    • Regular meetings to review progress