Evidence-based assessment/Step 10: Goal setting: Milestones and outcomes

=Process: Goal Setting for Milestones and Outcomes=

Overview
We want to keep our clients engaged with treatment and coming back until they achieve their goals. One of the best ways to do this is by working together to define goals that are achievable, measurable, and motivating to them. Having midterm and "final" goals, and having ways of tracking progress and sharing feedback session-by-session, all are ways of enhancing engagement, reducing drop-out ("premature termination"), and improving outcomes.

Rationale
Good assessment – combining formulation and detailed, personalized feedback--may be an effective intervention in its own right. A meta-analysis of 17 published studies found that assessment-as-intervention produced outcomes significantly better than the comparison group, with a Cohen’s d of .4, or roughly 2/3 of the assessment group having a better outcome than the average outcome for the comparison group not receiving the detailed feedback. Though the bulk of the studies were done with adults, the principle of providing detailed, personalized feedback is likely to generalize to family interventions; it also is a core element of coaching in sports as well as life skills. Regular feedback also reduces drop out, or premature termination.

Clinically significant change
Then standard rules of thumb would apply: 1.96 or greater would be 95% likely to reflect real change (i.e., occurring by chance less than 5% of the time), and 1.65 would be 90% likely. Others have suggested also reporting the 95% and 90% critical values in raw score metrics, as they would be easier for clinicians to use. Rather than observing that our client’s score on the Beck Depression Inventory decreased by 11 points, then dividing that change score by the standard error of the difference (4.8 points, per data reported in the technical manual) to get an RCI of 2.24, it is easier for a clinician to know that given the precision and stability of the BDI, changes of 8 points (rounding up from 7.9, the product of 1.65 and 4.8) are enough to be 90% confident and 10 (rounding up, not rounding off, per Jacobson’s recommendations) are 95% confident that they reflect a real pattern and not measurement error.

Convenience is another barrier. It is not realistic that busy clinicians would hunt down technical parameters and then perform calculations on a case-by-case basis in real time. For clinical significance to be feasible to measure in practice, the benchmarks and thresholds would need to be calculated ahead of time for the measures likely to be used (Step B), and ideally programmed into an Excel spreadsheet or Google Doc so that the clinician could just enter the score and get the results. Then, the baseline scores could be used to generate the targets for change. If a case started with an Externalizing score of T = 80, then the standard error of the difference score sets a change of 7 points as 90% likely to reflect real change (RCI > 1.65), or 8 points as 95% likely. Thresholds would be Back = 70, Closer = 56, and Away = 37 to 42 (depending on the age and sex of the youth). A way of explaining this to the family could be: “We definitely want our treatment to reduce her aggressive and rule-breaking behaviors. After some sessions, we should see progress if treatment is helping. The problem won’t be all gone, but if we asked you to fill this out as a progress check, and we saw the score drop by at least 7 points, we would be confident that treatment was helping. After some more sessions, if we got the score to less than 70, that would be another milestone that showed real progress. Youths her age show some of these behaviors, too, for all sorts of different reasons, and a score lower than 70 would be back in that ballpark of behavior, although still at the high end. With more sessions, we would like to get to the point where the score might drop further, below a 56. At that point, things would not be perfect, but the types of behaviors you would be seeing would be more typical of most teens, and not as similar to those who are coming to a clinic. If you work hard and things go very well, then scores below Away = 38 would indicate spectacular success, where they are radically different from where you started.”

Good assessment – combining formulation and detailed, personalized feedback--may be an effective intervention in its own right. A meta-analysis of 17 published studies found that assessment-as-intervention produced outcomes significantly better than the comparison group, with a Cohen’s d of .4, or roughly 2/3 of the assessment group having a better outcome than the average outcome for the comparison group not receiving the detailed feedback. Though the bulk of the studies were done with adults, the principle of providing detailed, personalized feedback is likely to generalize to family interventions; it also is a core element of coaching in sports as well as life skills. Regular feedback also reduces drop out, or premature termination.

Where to find copies of measures and supporting details
There are several websites that help gather copies of measures and information about them.

PROMIS. The NIH funded the development of the PROMIS outcome measures. These cover a wide range of health issues, including depression, anxiety, sleep, and others. The PROMIS measures currently are free to use as paper & pencil versions, and there is an iPad version with a licensing fee. The PROMIS measures are one of the first places where researchers are publishing MID benchmarks (e.g., Thissen et al., 2016). As of March 2017, these do not have the Jacobson & Truax type benchmarks for Away, Back, and Closer, nor are the SEs of the difference score usually reported.

Society for Clinical Psychology.

EBA on Wikipedia.

California Clearinghouse.