---
title: "Getting Started with `integrity`"
author:   Sol Libesman, David Nguyen, Dario Strbenac, Jie Kang, Anna Lene Seidler, Kylie Hunter <br>
        The University of Sydney, Australia.
output:
  html_document:
    toc: true
vignette: >
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteIndexEntry{An Introduction to the integrity Package}
---

<style>
body {
  color: #263238;
}

h1, h2, h3, h4 {
  color: #17324d;
}

h2 {
  border-bottom: 2px solid #d9e2ec;
  padding-bottom: 0.25em;
  margin-top: 1.5em;
}

h3 {
  margin-top: 1.4em;
}

h4 {
  border-left: 4px solid #2f80ed;
  padding: 0.35em 0 0.35em 0.75em;
  background: #f5f9ff;
}

table {
  width: 100%;
  margin: 0.75em 0 1.5em 0;
}

thead {
  background: #edf2f7;
}

th, td {
  padding: 0.45em 0.6em;
  vertical-align: top;
}

blockquote {
  border-left: 4px solid #7b8794;
  color: #52606d;
  background: #f7f9fb;
  padding: 0.5em 1em;
}
</style>


## Package overview

Increasing concerns about the trustworthiness of research have prompted calls to scrutinise studies' individual participant data (IPD), that is, de-identified raw line-by-line data for each participant in a study.

`integrity` was developed to support application of the IPD Integrity Tool [(Hunter et al. 2024A)](https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1738). It enables structured and transparent assessment of the integrity and trustworthiness of randomised controlled trials (RCTs) using IPD and informs decisions about  whether RCTs should be included in evidence synthesis or considered suitable for publication. Further information may be found about the development of the tool [here - (Hunter et al., 2024B)](https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1739). 

If you use our package please cite:

&ensp; &ensp; Hunter KE, Aberoumand M, Libesman S, Sotiropoulos JX, Williams JG, Aagerup J, Wang R, Mol BW, Li W, Barba A, Shrestha N. Webster AC, Seidler AL. The Individual Participant Data Integrity Tool for assessing the integrity of randomised trials. *Research Synthesis Methods*. 2024 Nov;15(6):917-39.


## How to use the package 

Each step of the workflow is illustrated using a case study on umbilical cord management at preterm birth, based on a de-identified and altered data set from the [iCOMP study](https://doi.org/10.1016/S0140-6736(23)02468-6). The main goal was to determine the optimal umbilical cord management strategy at preterm birth, such as milking or delayed cord clamping.

You can download the editable R Markdown (`.Rmd`) vignette to run the case study on your own computer or adapt it to generate a report using your own data.

1. Open the [integrity CRAN package page](https://CRAN.R-project.org/package=integrity). Under **Vignettes**, find **An Introduction to the integrity Package** and right-click its **source** link, then select **Save link as...** (or the equivalent option in your browser).
2. Choose the relevant download location and name the file `my_integrity_report.Rmd`. Ensure the filename ends in `.Rmd`, rather than `.Rmd.txt`. If your browser displays the source as plain text, use **Save As** to save it with the `.Rmd` extension.
3. In RStudio, select **File > Open File** and open `my_integrity_report.Rmd` from your chosen download location. Follow the one-time installation instructions in **Step 1: Data loading**, then replace the bundled example-data import with an import of your own dataset. Use the full path to your data file, or place it beside the `.Rmd` file and use its filename. On Windows, use forward slashes in R file paths, for example `C:/Users/YourName/Desktop/my_data.xlsx`.
4. In **Step 2: Data preparation**, edit `dataset_info` so its column names, variable types, and expected values match your dataset. Remove optional metadata entries that do not apply. You can also update the report title, author, and case-study text for your study.
5. Save your changes and click **Knit** in RStudio to generate an HTML report. Review the status tables, caution messages, and plots before interpreting the results.


## Step 1: Data loading

Before knitting this report for the first time, run the following command in the RStudio Console to install the package and the packages used by this example. This installation command is not run automatically when you knit the report. Install any additional packages needed to import your own data separately.

```{r, eval=FALSE}
install.packages(c("integrity", "rmarkdown", "knitr", "readxl"))
```

Load the integrity package into R each time you run the report.

```{r}
library(integrity)
```

Next, import the data set you wish to examine into R. There are a variety of functions in R or CRAN packages to do this:

- `read.csv` and `read.table` functions to import comma-separated and tab-separated text files.
- `read.sas` for SAS, `read.sav` for SPSS and `read_dta` for STATA in the CRAN package [haven](https://haven.tidyverse.org/).
- `read_excel` function for Microsoft Excel in the CRAN package [readxl](https://readxl.tidyverse.org/).


Case study: The altered iCOMP case study is loaded with the `integrity` package. The data are in a Microsoft Excel file.

```{r, warning =F}

library(readxl)
examplePath <- system.file("extdata", "dataset.xlsx", package = "integrity")
dataset <- read_excel(examplePath, sheet=1)
dataset[1:5, ]
```


In the tibble above, the sample identifiers can be seen (`infant_ID`), as well as the date of randomisation (`rand_date`) and the first few clinical covariates. 


## Step 2: Data preparation 


The `integrity` tool requires a list of trial metadata that maps the variables in your dataset to the relevant integrity checks. For each element you supply, specify the corresponding column name(s) in your dataset:

- `participantID`: The name of the column which corresponds to the unique participant identifier (this variable is mandatory).
- `enrollment`: lists the names of three columns corresponding to `start` (date of first participant enrollment), `randomisation` (date of participant randomisation) and `end` (date of the last participant enrollment).
- `baseline`: lists named `dichotomous`, `polytomous`, `numeric` are for specifying the column name(s) of
the column(s) which correspond(s) to baseline measurements.
- `intervention`: the name of the column specifying the intervention or group allocation for each individual (required for comparisons between study groups).
- `outcome`: lists named `common` and `rare`, with sublists named by `dichotomous`, `numeric` or `polytomous`, containing the names of columns of those data types for outcomes assessed.
- `correlated`: A named list of variable pairs expected to be correlated, with two column names per pair.
- `unexpected`: A named list of column names with values that are not expected to be seen. `days` is a special sublist and applies to date columns, which are converted into days of the week before comparison. It must have two elements: `names`, which are the unexpected day names, and `locale`, which is the locale of the unexpected day names specified.

Only `participantID` is strictly required. Supply `enrollment`, `baseline`, `intervention`, `outcome`, `correlated`, and `unexpected` when available. If your dataset does not contain a variable, remove its optional metadata entry or comment it out by placing `#` at the start of the relevant line(s). For example, no polytomous baseline variables exist in the case study and so that entry is commented out in the code below. Checks that require the omitted variable will be skipped.

If date columns are imported as text, convert them to R dates or date-times before running the checks. For example, `as.Date(x, format = "%Y-%m-%d")` converts text dates such as `2026-10-12`; adjust the format to match your data. Randomisation-date plots require a recognised date or date-time column.

The variable types and expectations need to be defined before running the checks. The example below defines this metadata as a named list written directly in R.

Coding the list directly in R as below, is the simplest option for users already working inside an R script or an R Markdown document, because the metadata can be written directly next to the analysis code. The R code below may be used as a template and altered based on relevant variables in a new dataset.

```{r}
dataset_info <- list(
  participantID = "infant_ID",
  enrollment = list(
    start = "enrol_start",
    randomisation = "rand_date",
    end = "enrol_end"
  ),
  baseline = list(
    dichotomous = c("sex"), # add more variables if needed e.g., c("sex", "respiratory_support")
    #polytomous = c("ordinal_or_nominal_variable"), # no polytomous baseline variables in this data set so it's commented out 
    numeric = c("mat_age", "GA_weeks", "birthweight")
    # can add polytomous variables if needed
  ),
  intervention = "treatment_cat",
  outcome = list(
    common = list(
      dichotomous = c("IVH", "NEC"),
      polytomous = c("CLD"), # if certain variable types don't exist, just delete the relevant line.
      numeric = c("hospital_days")
    ),
    rare = list(
      dichotomous = c("inf_death") # add more variables if needed e.g., c("inf_death", "severe_IVH")
    )
  ),
  correlated = list(
    timeAndSize = c("GA_weeks", "birthweight")
  ),
    # Example with two variable pairs (replace names with your own numeric columns):
  # correlated = list(
  #   timeAndSize = c("GA_weeks", "birthweight"),
  #   heightAndWeight = c("height", "weight")
  # ),
  unexpected = list(
    days = list(
      names = c("Saturday", "Sunday"),
      locale = "C"
    ),
    mat_age = c("less than 10", "greater than 50"),
    GA_weeks = c("less than 22", "greater than 37")
  )
)
```

In the `unexpected$days` section, `locale` controls the language used when R converts dates into weekday names. `locale = "C"` is the most likely option to use because it returns standard English weekday names such as `Saturday` and `Sunday`, which usually match the values entered in `names`. Other locale values are possible if your system or dataset uses a different language or naming convention, but `"C"` will usually be the safest default.


## Step 3: Running integrity checks


Simply provide the data frame from the trial you are checking (in this case `dataset`) and data information list (`dataset_info`) to `run_checks`. The function first performs automated data checking and cleaning to check whether the nominated variables are present and report missing columns. The function will also convert columns nominated as factors into factors where required, and remove any columns containing only missing values.

```{r}
result <- run_checks(dataset, dataset_info)
names(result)
```

This creates a list of result objects, including overall check tables, detailed per-variable tables for selected checks, plots, and summary tables. The output for each item below should be reviewed consecutively and rated using the decision guide and rating sheet [(found here - Hunter et al. 2024A)](https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1738)

## Step 4: Reviewing results by integrity domain

The sections below present the output from the integrity  `run_checks` function, split under each domain and item.

### Domain 1: Unusual or repeated data patterns

**Item 1.1**: Repeating patterns within baseline variables

This item involves sorting and visually inspecting baseline variables for repeating patterns that may indicate rows have been copied and pasted. Look for values that recur at regular intervals. Rare or unusual entries can be particularly informative, for example, an unusual value that appears every 11 rows.

The plots below support visual inspection of continuous and categorical baseline variables. Participants are ordered by randomisation date when available, and points are coloured by study group. If no usable randomisation date is available, participants are plotted in dataset row order. In this case, interpret patterns cautiously, as they may reflect how the dataset was sorted. For continuous variables, values should generally display a random scatter across participants, with no obvious repeating sequences or regular spacing. For categorical variables, categories should also appear randomly distributed. Large, recurring gaps between participants with the same category may indicate a non-random pattern, although such gaps can occur naturally when a category is rare. Points are coloured by study group to help visualise patterns both within and between groups. Consider whether the overall distribution appears genuinely random or whether repeating sequences, regular intervals, unusual clustering, or other systematic patterns suggest potential data irregularities.

```{r}
item_1_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_1, row.names = FALSE)
baseline_pattern_plots <- grep("^Repeated Baselines Within Variables - ", names(result[["images"]]), value = TRUE)
for(plot_name in baseline_pattern_plots) {
  print(result[["images"]][[plot_name]])
}
```

**Item 1.2**: Repeating data patterns across baseline variables

This item looks for duplication across participants, e.g. do all participants with a height of 180cm have the same weight? Duplicate entries for baseline variables are listed below (if there are none, no input will be printed).  

```{r}
item_1_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_2, row.names = FALSE)
```

**Item 1.3**:  Repeating data patterns across baseline variables and rare variables. 

```{r}
item_1_3 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.3", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_3, row.names = FALSE)
```

**Item 1.4**: Bias in the terminal (rightmost) digits.

This item plots the terminal digit for the selected continuous variables (avoid variables that tend to be rounded or that lack precision). Each variable has its own y-axis scale. Inspect the bar charts for biased or unexpected distribution.


```{r}
item_1_4 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.4", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_4, row.names = FALSE)
if("Terminal Digits" %in% names(result[["images"]])) {
  result[["images"]][["Terminal Digits"]]
}
```

### Domain 2: Baseline characteristics

**Item 2.1**:  Excessively homogeneous distribution of binary baseline variables, i.e. loss of independence between consecutive observations

The two-sided Wald-Wolfowitz runs test checks whether binary baseline values have unusually few or unusually many runs (consecutive blocks of the same value). The detailed table reports observed and expected runs, p values, and the test method. When either category has 10 or fewer observations, an exact conditional test is used: the two-sided p value is twice the smaller inclusive tail probability, capped at one. Otherwise, the test uses a normal approximation without continuity correction.

The test automatically orders participants by randomisation date. If no usable randomisation date variable is available, the test uses dataset row order and results may be invalid. A significant result requires contextual review, particularly for rare categories, time trends, and multiple tests; it does not establish an integrity problem by itself.

```{r}
item_2_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_1, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.1"]])) knitr::kable(result[["detail_tables"]][["2.1"]], row.names = FALSE)
```

**Item 2.2**: Excessive imbalances between groups in continuous baseline variables.

Evaluates mean and standard deviation for key prognostic factors that are continuous, split by treatment group. 

```{r}
item_2_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_2, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.2"]])) knitr::kable(result[["detail_tables"]][["2.2"]], row.names = FALSE)
```

**Item 2.3**: Excessive imbalances in baseline categorical variables between groups.

This item assesses whether counts of baseline categorical variables are significantly different (p<0.05) between groups. 

```{r}
item_2_3 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.3", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_3, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.3"]])) knitr::kable(result[["detail_tables"]][["2.3"]], row.names = FALSE)
```

**Item 2.4**: Significant difference in variance of continuous baseline variables between groups.

This item uses Levene's test, which checks whether there is a significant difference in variability between groups.

```{r}
item_2_4 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.4", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_4, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.4"]])) knitr::kable(result[["detail_tables"]][["2.4"]], row.names = FALSE)
```

### Domain 3: Correlations

**Item 3.1**: No association between variables known to be highly correlated.

This item plots the correlation between selected continuous variables and calculates the Pearson correlation coefficient (R) and associated p value. Check whether expected correlations are present. 

```{r}
item_3_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "3.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_3_1, row.names = FALSE)
correlation_plots <- intersect(names(dataset_info[["correlated"]]), names(result[["images"]]))
for(plot_name in correlation_plots) {
  print(result[["images"]][[plot_name]])
}
```

### Domain 4: Date violations

**Item 4.1**: Individual enrolment dates do not fit within study start and end dates.

This item examines whether randomisation dates for each individual fall within the enrolment period. 

```{r}
item_4_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "4.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_4_1, row.names = FALSE)

item_4_1_dates <- result[["detail_tables"]][["4.1"]]
if(!is.null(item_4_1_dates)) {
  knitr::kable(item_4_1_dates, row.names = FALSE)
}
```

**Item 4.2**:  Dates (or visits) are not in logical order.

Requires study-specific repeated visits or event-date variables; for example, a follow-up date occurring before enrollment.

```{r}
item_4_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "4.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_4_2, row.names = FALSE)
```

### Domain 5: Patterns of allocation

**Item 5.1**:  Non-random allocation patterns: plot.

The plot below shows the cumulative number of allocated participants to each treatment arm by date of randomisation. We expect the cumulative number of randomised participants in each group to be similar if 1:1 allocation is used. Assess whether cumulative lines for treatment groups deviate from each other drastically.
Note: the graphs will only appear when the date of randomisation is provided.


```{r}
item_5_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "5.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_5_1, row.names = FALSE)
if("Cumulative Allocation" %in% names(result[["images"]])) {
  result[["images"]][["Cumulative Allocation"]]
}
```

**Item 5.2**:  Non-random allocation patterns: statistical test

This item evaluates randomness of allocation using two approaches: a runs test and a chi-squared test comparing observed adjacent intervention runs with the expected number under random allocation. A statistically significant result (p<0.05) from either test may be indicative of an issue with randomisation.

```{r}
item_5_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "5.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_5_2, row.names = FALSE)
item_5_2_test <- result[["detail_tables"]][["5.2"]]
if(!is.null(item_5_2_test)) {
  knitr::kable(item_5_2_test, row.names = FALSE)
}
```

**Item 5.3**: Unexpected imbalance in randomisation day of week.

The table below reports two chi-squared tests: one assessing whether randomisation is distributed evenly across weekdays overall, and one assessing whether randomisation day differs by intervention group. The graph below shows the number of participants randomised on each day of the week by group. We expect numbers to be balanced between groups for each weekday, and fewer enrolments on the weekend for non-urgent interventions. 
Note: the graph will only appear when the date of randomisation is provided.


```{r}
item_5_3 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "5.3", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_5_3, row.names = FALSE)
item_5_3_test <- result[["detail_tables"]][["5.3"]]
if(!is.null(item_5_3_test)) {
  knitr::kable(item_5_3_test, row.names = FALSE)
}
if("Days" %in% names(result[["images"]])) {
  result[["images"]][["Days"]]
}
```

### Domain 6: Internal inconsistencies

**Item 6.1**: Inconsistent or illogical values across variables within individual participants.

Derive logic rules for each variable to be collected, e.g. date of hospital discharge = date of admission + days in hospital; if number of transfusions ≥1, then any transfusion = yes. Incorporate these rules into the package so that any breaches are displayed in the output

```{r}
item_6_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "6.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_6_1, row.names = FALSE)
```

### Domain 7: External inconsistencies

**Item 7.1**: IPD do not correspond to publications or reports.

The table below shows summary statistics for each variable provided in the IPD dataset, e.g. mean, median, range, etc. Manually cross‐check these against any published trial reports, including appendices and supplements. Record any inconsistencies identified, for example, discrepancies in summary variable values between IPD and publication, inclusion of participants in IPD that do not meet eligibility criteria in publication, published variables that are missing from IPD dataset. If data are provided for excluded participants, check whether reasons for exclusion are consistent with publication.

```{r}
item_7_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "7.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_7_1, row.names = FALSE)
if(!is.null(result[["summary_table"]])) {
  result[["summary_table"]]
}
```

### Domain 8: Plausibility of data

**Item 8.1**: Too few missing data or missing data are overly similar between groups.

The table below shows missingness by intervention group for outcome variables, including the percentage missing in each group.

```{r}
item_8_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "8.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_8_1, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["8.1"]])) knitr::kable(result[["detail_tables"]][["8.1"]], row.names = FALSE)
```

**Item 8.2**: Implausible event rates: outcomes and demographics.

The table below shows events and totals for dichotomous baseline variables and dichotomous common and rare outcomes, split by intervention group.

```{r}
item_8_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "8.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_8_2, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["8.2"]])) knitr::kable(result[["detail_tables"]][["8.2"]], row.names = FALSE)
```

## Computing Environment

This vignette was executed on the following computing system:

```{r}
sessionInfo()
```