This page provides a short summary of our summative evaluation results of the SBIRT Core Training Activity.
Primary care providers (PCPs) and other health care professionals should provide Screening, Brief Intervention, and Referral to Treatment (SBIRT) for patients with active or potential substance use problems. To address a gap in training opportunities, Clinical Tools, Inc. created the SBIRT Core Training Activity, a new web-based skills training for PCPs on how to provide SBIRT for substance use problems. The activity aims to improve clinical practice and patient outcomes regarding substance use. Needs analysis, best practice guidelines, and expert review from the American Society of Addiction Medicine (ASAM) guided development of the training curriculum.
The training activity was activated in April 2011 for evaluation purposes. Between July-September 2011, the summative evaluation of the activity, a controlled comparison pre- to post-training of assessment performance, was completed by 33 intervention group participants and 41 control group participants, all of whom were primary care physicians.
Results
Pre-/Post- Knowledge
Results showed significant improvements pre-training to post-training for the intervention group in assessment scores for knowledge (p ≥ 0.009).
| Pre | 2nd measure | Difference | p value | 3rd measure | |
| Intervention (n=33) | 74.50% | post-training 84.90% | 10.50% | p≥0.01 | followup (n=12) 74.9% |
| Control (n=41) | 73.50% | post-wait period 78.40% |
4.90% | p≥0.09 | post-training (n=15) 85.8% (p≥0.02) |
Pre-/Post- Clinical Skills
Assessment scores for clinical skills on a patient record improved significantly from pre- (48.1%) to post (53.8%) for the intervention group (p≥0.04).
Patient Note Section (Maximum Possible Points)Mean Pre-Score (%)Mean Post- Score (%)Mean % Change
| Significant Changes in Patient Note: Intervention (n=31) | |||
| Differential Diagnosis (1) | 62.5 | 80.8 | 18.3 |
| Labs/Diagnostic Tests/Assessments (2) | 28.1 | 39.9 | 11.8 |
| Brief Intervention (6) | 55.9 | 71.5 | 15.6 |
| Overall mean (26) | 48.1 | 53.8 | 5.7 |
| p value | p≥0.04 (positive change for Intervention group) | ||
Behavior
Following training, nearly all intervention participants intended to increase the following brief intervention and followup behaviors:
- Brief interventions for tobacco (97% of participants)
- Brief interventions for alcohol (94%)
- Brief interventions for drugs (97%)
- Followup after substance abuse treatment (94%)
At the 3 wk followup, the percentage of intervention group participants planning considerable or significant change went up from pre- and post- values for five behaviors. Examples:
- “Follow-up with and reassess patients who receive treatment for substance abuse or tobacco use”
- “Screen for illicit drug use”
Educational value and Satisfaction
Educational value. The majority of participants agreed that the activity improved:
- Knowledge (100%)
- Competence (91%)
- Performance (70%) and
- Patient outcomes (61%)
There was strong agreement or agreement by 100% of participants that the activity would impact their practice and that the activity promotes improvement in healthcare.
Satisfaction. Ratings ranged from a mean of 4.2 to 4.6 on a scale of 1 to 5 for agreement with six satisfaction statements (where 1=Strongly Disagree, 5=Strongly Agree).
| Amount of Agreement | Satisfaction Statement | Rate of Agreement/Strong Agreement |
| Statements having the highest agreement | “This activity was presented objectively and was free of commercial bias.” “Overall, this was a useful learning experience.” |
100% |
| Statement having the lowest agreement | “I was able to navigate the website easily.” *Note: Navigation problems during beta testing have been resolved. | 88% |
