About RoB-Σ
RoB-Σ · Risk of Bias Sigma - a comprehensive sum of bias assessment methods
The Σ (capital Sigma) is the mathematical symbol for summation - it represents summing up all aspects of a complex problem into one integrated result. In RoB-Σ, this reflects the tool's function: it does not implement a single instrument in isolation, but consolidates six internationally validated risk-of-bias frameworks (RoB 2, NOS, ROBINS-I, QUADAS-2, QUIPS, AMSTAR 2) into one guided platform - the sum of methodological bias assessment across all major study designs. Every domain judgment, override, and rationale is tracked in a single unified audit trail. The Σ is also the signature of the InsilicoΣ lab, identifying tools that sum up all relevant analytical dimensions rather than addressing them piecemeal.
RoB-Σ is a guided, user-friendly tool for assessing risk of bias in studies included in systematic reviews. It supports six internationally recognized assessment instruments.
Supported Instruments
| Study Type | Tool | Judgment Scale | Reference |
|---|---|---|---|
| RCT | RoB 2 | Low / Some concerns / High | Sterne JAC et al. BMJ 2019; 366:l4898 |
| Cohort | NOS (Cohort) | Low risk / Unclear risk / High risk | Wells GA et al. Ottawa Hospital Research Institute |
| Case-control | NOS (Case-Control) | Low risk / Unclear risk / High risk | Wells GA et al. Ottawa Hospital Research Institute |
| Cohort, Case-control, Non-randomized intervention | ROBINS-I | Low / Moderate / Serious / Critical / No information | Sterne JA et al. BMJ 2016; 355:i4919 |
| Diagnostic accuracy | QUADAS-2 | Low / High / Unclear | Whiting PF et al. Ann Intern Med 2011; 155(8):529-536 |
| Prognostic | QUIPS | Low / Moderate / High | Hayden JA et al. Ann Intern Med 2013; 158(4):280-286 |
| Systematic review | AMSTAR 2 | High / Moderate / Low / Critically low | Shea BJ et al. BMJ 2017; 358:j4008 |
How It Works
- Create a project for your systematic review
- Add studies and select the appropriate assessment instrument
- Answer guided questions domain by domain - the algorithm computes judgments automatically
- Review and override judgments with justification if needed
- Generate figures (traffic light plots, summary bar charts) for your manuscript
- Export results as CSV, Excel, or PDF
Algorithm Methodology
RoB-Σ implements each instrument's published decision-tree algorithm rather than a generic counting rule. Polarity, conditional gating, and reverse-coded questions are handled per the source paper. Algorithm output is a guide - every domain and the overall judgment can be overridden by the assessor with a recorded justification.
| Instrument | Domain rule source | Overall rule |
|---|---|---|
| RoB 2 | Sterne 2019 Box 1: per-domain decision rules with reverse-coded questions (Q1.3, D4, etc.) and routing questions (Q2.1/2.2/4.3) explicitly handled. | Sterne 2019 Box 2: worst-case across D1-D5; reviewer is alerted when multiple domains carry "Some concerns". |
| ROBINS-I | Sterne 2016 Table 5: D1 gating (Q1.1 No -> Low directly), Q1.6 post-intervention adjustment -> Serious, Q1.4 inadequate control -> Serious, etc. for D2-D7. | Sterne 2016 Table 6: worst-case across the seven domains. "Critical" is never assigned automatically - reserved for the manual override per Sterne's guidance. |
| QUADAS-2 | Whiting 2011: signalling questions uniformly Yes = favourable. Per- domain risk + separate applicability concern. | Worst-case across the four domains for risk and applicability. |
| QUIPS | Hayden 2013 does not prescribe a counting algorithm. RoB-Σ uses a transparent heuristic (all Yes -> Low; >=50% No -> High; else Moderate) that the reviewer is encouraged to override. | Worst-case across the six domains. |
| NOS (Cohort / Case-Control) | Per Wells: stars awarded per item; comparability stars must be earned sequentially (the second star cannot be granted without the first). | Star totals mapped to Low/Unclear/High using AHRQ thresholds (>=7/4-6/<4). Note: the Cochrane Handbook v6.4 ch.25 cautions against dichotomising NOS totals; reviewers should use the override field where the pattern of stars matters more than the count. |
| AMSTAR 2 | Shea 2017: per-item Yes / Partial yes / No / No meta-analysis. "Partial yes" is treated as a non-critical weakness (not a critical flaw). "No meta-analysis" responses on Q11/Q12/Q15 are excluded from the weakness count. | Shea 2017 Table 2: thresholds on critical and non-critical weaknesses to assign High / Moderate / Low / Critically low confidence. |
Implementation lives in
rob_sigma/services/judgment_engine.py. Source rules are cited in
the docstring of each domain function. When the engine is updated (current
version: 2026-05-12-decision-trees),
legacy assessments can be refreshed with
manage.py recompute_judgments while preserving every user
override.
Key References
- RoB 2: Sterne JAC et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366:l4898
- NOS: Wells GA et al. The Newcastle-Ottawa Scale for assessing the quality of nonrandomised studies in meta-analyses. Ottawa Hospital Research Institute.
- ROBINS-I: Sterne JA et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355:i4919
- QUADAS-2: Whiting PF et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155(8):529-536
- QUIPS: Hayden JA et al. Assessing bias in studies of prognostic factors. Ann Intern Med 2013; 158(4):280-286
- AMSTAR 2: Shea BJ et al. AMSTAR 2: a critical appraisal tool for systematic reviews. BMJ 2017; 358:j4008