Evidence-based assessment/Step 11: Progress and process measures

=Process Phase: Progress and Process Measures=

Overview
The most fine-grained level of assessment tracks change in between sessions. In the journey of therapy, this is the GPS measuring progress towards goals (how many kilometers covered?). Daily or session-level measures act as the speedometer, providing a sense of how rapidly things are progressing, or whether they are stalled or moving backwards. Practicing without systematic assessment is akin to barreling blind into space – unlikely to help you arrive at your destination on time and without significant detours. Mapping out your route before you leave and checking progress against that plan allows for recalculating when obstacles are encountered and an ongoing conversation between therapist and client about what the ultimate destination is and how best to get there.

An analysis of the treatment manuals for a set of ESTs observed that all of them included some form of progress tracking, whereas treatment as usual often tracks progress informally and impressionistically, if at all. The techniques used here vary widely across interventions, from daily report cards completed by teachers, to sticker charts and token economies, or three and five column charts in CBT, diary cards in DBT, and daily life or mood charts for mood disorder, including several free examples that have been customized for children. These regular progress measures increase the effectiveness of the treatment, the same as how weigh-ins give a diet more traction. Giving clinicians very simple feedback about whether the client is making good progress, stalled, or worsening changes therapy outcomes, including a reduction in premature termination and better retention of difficult cases, as well as faster planned termination of cases who responded well. Streamlined assessment tracking a few top problems on a session-by-session basis is both feasible and highly sensitive to treatment effects with youths and families. Figure 5 shows an example of plotting session level tracking using the Top Problems type of scaling, juxtaposed with tracking the session progress. The more frequent schedule of measurement makes these data even more useful as a harbinger of progress or setbacks, and makes more rapid course correction possible. These data also can signal appropriate times to transition to tapering and planned termination, as well as continued monitoring to maintain gains. Tracking process is where the traditional assessment measures are weakest. None of the commonly taught and used measures are suitable for weekly, let alone daily, repetition. This is a niche ripe for innovation. This also is the place where technology is most rapidly transforming assessment. Mood charts have migrated from paper and pencil to online, and thence to text messaging and smartphone apps. Daily report cards now can be emailed or texted to teachers, and texting also makes ecological momentary assessment of mood and energy feasible. Apps can now use phones as actigraphs, providing objective measures of energy and sleep. Wearable devices such as the FitBit and smart watches also add to the potential for passively tracking footsteps, other motor activity, heartrate, and peripheral temperature. Apps can even monitor the cadence and intonation of speech to assess for depression or hostility.

Sleep tracking example. Figure 6 shows two forms of sleep data that an inexpensive smartphone app generates. Our client would install the app on her phone, turn it on as she went to bed, and put the phone on the mattress next to her. The phone’s gyroscope detects her movements during the night, and uses algorithms to estimate when she fell asleep, when she woke, and even sleep stage (light sleep, deep sleep, and REM). The sleep onset and duration measures correlate surprisingly well with established criterion measures (REM not so well yet). The phone methodology creates a low burden way for her to track sleep, revealing periods of sleep disruption that then can be discussed in treatment.

Monitoring social media can also gather information about risky behaviors or suicidal ideation, and feasibly could lead to passive assessments of other psychological constructs that do not require any additional effort on the part of the patient (see also the Durkheim Project, developed for the US military). These new information systems are still getting patched into the dashboard guiding therapy, and passive monitoring raises a new set of ethical considerations that need attention. Is informed consent needed before monitoring public Facebook pages for cues of risk? Is the consent of the parent required even though the information is public? Is the assent of the youth required? The questions, and workable solutions, will come into focus quickly in the next few years.

The ongoing assessment also makes course-correction possible, improving the chances of a good outcome and perhaps reducing the risk of premature drop-out as well as preventing harm. When clinicians were randomly chosen to receive ongoing progress feedback, only 13% of clients deteriorated versus 21% when the clinicians did not get the progress monitoring; and 35% of clients made significant gains in the progress monitoring condition, versus only 21% in the treatment as usual arm. These rates convert into a Number Needed to Treat of 7.1, meaning that adding routine progress monitoring will create one more case with significant gains per seven cases added. More provocatively, continuing with treatment as usual results in a Number Needed to Harm of 12.5, meaning that roughly one in a dozen cases would have a deteriorating course that could have been prevented by adding progress monitoring. Weekly feedback to the therapist produces significantly faster response, with increased doses of feedback enhancing the patient response. Measurement-based care increases the rate and speed of response, roughly doubling the rate of remission achieved with pharmacotherapy of major depression in a randomized trial versus assessment as usual. It also is demonstrating good ability to identify both youth and adult cases at risk of dropping out, outperforming clinician judgment. This makes it possible to have an early warning system that could prompt the clinician to actively re-engage and discuss things with the patient – a win for both parties when it leads to better outcomes.

Steps to put into practice
There are several options about how to do this:
 * Jacobson and colleagues developed a two part framework for clinically significant change, combining a "Reliable Change Index" (RCI) with a set of benchmarks defined by norms based on people with and without a clinical condition. Several technical refinements have been suggested, but the Jacobson model remains one of the most well known approaches. It also is a worthy example of how to use psychometric and nomothetic data to set goals and measure outcomes for individual cases.


 * Brinley plots


 * Graphical approaches


 * Time series analysis