Types of Tailoring

Gain and loss-framed message content has received significant attention in the general health promotion literature and is the most studied framing paradigm for digital interventions across health domains [4, 14, 15]. Overall, these studies indicate that gain-framed messages modestly outperform loss-framed messages, but differential outcomes are often moderated by factors such as readiness to change or the target health behavior. However, message framing has not been examined specifically when delivered via text messaging.

Similarly, while message tailoring has been studied in the general health promotion literature, there has been virtually no direct examination or comparison of tailored vs. untailored mobile interventions for any behavior. Tailoring can refer to personal relevance (e.g. name) or tailoring to a specific target behavior-related characteristic (e.g. severity, self-efficacy). Studies have generally suggested that content-tailored interventions yield better outcomes than untailored interventions, especially when content of the untailored intervention has no relevance to the recipient’s personal experiences [16–18]. An additional form of tailoring entails adapting the intervention content to the current needs of the individual, which we refer to as adaptive tailoring. While the literature on comparing adaptively tailored interventions to statically tailored interventions is lacking, some research has shown that tailored computer-based interventions that adapt to the recipient’s progress (or lack thereof) outperform static tailored interventions [18].

The primary aim of this pilot study was to advance the literature on SMS interventions for PD by examining the effects of gain-framed (GF), loss-framed (LF), statically tailored (ST) and adaptively tailored (TA) SMS programs compared to weekly mobile assessment only (MA) on alcoholic drink reduction over a 12-week period in individuals seeking assistance on the internet to reduce their alcohol consumption to safe levels. To achieve these aims, we conducted an exploratory, single-blind, randomized controlled pilot study comparing effect sizes of weekly self-tracking mobile assessment only to the different daily messaging groups on reductions in weekly drinks, weekly heavy drinking days, number of days without drinking and drinking consequences over a 12-week period. Outcomes are designed to generate inform the development and testing of future messaging studies to reduce problem drinking.

Participants and Procedure

Participants were recruited between April 2014 and January 2015 through online alcohol screening and help-seeking sources, such as AlcoholScreening.org and Moderation.org. Advertisements offered individuals worried about their drinking the opportunity to screen for a research study to find out if texts could help them manage their alcohol consumption. Prospective participants were directed to the study website, which offered basic information about the study and a link to a brief screening survey in Survey Monkey. IP blocking once the web screening was completed ensured that participants could only complete the survey once from any given device. Individuals eligible to continue the screening process based on the survey were provided with the study’s email address to schedule a time to complete a 20-minute phone screening with study staff. Participants deemed eligible based on the screening call provided verbal assent and were emailed two links: one to a web-based consent form and consent form quiz, and another to the baseline survey. All participants electronically signed the consent form and any incorrect answers to the consent form quiz were reviewed to ensure comprehension. Ineligible participants were directed to online drinking resources, the SAMHSA Treatment Finder, and were offered referrals.

Participants were eligible for the study if they: (1) consumed at least 13 and 15 standard drinks per week for women and men, respectively; (2) were willing to reduce their drinking to non-hazardous levels by study completion; (3) were between the ages of 21–65; (4) owned a mobile phone and were willing to receive and respond to text messages; (5) were fluent in English; and (6) could read at the 8th grade level, as determined by the consent form quiz. Participants were excluded from the study if they: (1) drank more than 45 standard drinks per week; (2) demonstrated clinically severe alcoholism (evidenced by physical withdrawal symptoms or a history of serious withdrawal symptoms, such as hallucinations or seizures); (3) scored above a 12 on the Short Alcohol Withdrawal Scale [19]; (4) presented with a current substance use disorder for any substance other than alcohol, nicotine, or caffeine (defined as more than once weekly use in the past month); (5) used marijuana more than twice weekly in the past month; (6) self-reported or presented with a serious psychiatric illness as evidenced by a previous diagnosis of psychosis or bipolar disorder, inpatient treatment, medication for psychosis, or recent suicidality; (6) were already in alcohol treatment or expressed a desire to obtain additional substance abuse treatment while in the study; (7) reported a medical condition that precluded drinking any alcohol, pregnancy or a desire to become pregnant while in the study; (8) reported a desire to pursue either trial or long-term abstinence; or (9) demonstrated a lack of understanding of study protocol or reading difficulty as evidenced by a score of less than 7 out of 10 on the consent form quiz. The majority of eligibility criteria were assessed during the online self-report screening questionnaire. About halfway through the trial, we reduced the drinking criteria for eligibility from 21 and 24 standard drinks per week for women and men, respectively, to 13 for women and 15 for men due to the number of drinkers screened who did not meet the original threshold, but still reported problems resulting from their drinking. There were no differences at baseline in drinking severity between messaging groups. This study was approved by both the New York State Psychiatric Institute Institutional Review Board (IRB) and the Northwell Health Institutional Review Board (IRB) (S1 Fig), and all clinical investigation was conducted according to the principles expressed in the Declaration of Helsinki.

Study flow, reasons for exclusion, study assignment, and retention are presented in the Consort Flow Diagram (Fig 1). Within an 8-month period, 1,149 individuals completed the screening questionnaire and 661 were eligible for phone screening based on the initial web-screen. Of those, 287 completed a phone screening. In total, 189 individuals were deemed eligible for study participation based on the web- and phone- screenings, and 176 completed the consent form and baseline assessment. 19 participants were force-randomized into a group with no specific alcohol content based on their request to receive messages that did not include terms referring to alcohol and drinking. Because this small group was force-randomized, they are presented separately from the active messaging groups. We present the descriptive findings for this group in the results section. Of the remainder, 157 were randomized to a study condition after completing the consenting process and five were withdrawn within the first week of randomization. Of the five participants excluded from the final sample, one was excluded due to incoherent and inconsistent responses to baseline web-based assessments, one was withdrawn due to pregnancy, one was excluded due to a technology-based error that precluded the user from receiving the text messages, one was excluded for drinking too little at baseline to be included in analyses, and one was excluded as a “technology troll” (someone working for a company developing mobile apps for health behavior change) who mined the program’s content for a week before dropping out. Of the remaining 152 participants who completed the baseline assessment and were not withdrawn, seven were lost to contact prior to the week 12 assessment. The final randomized sample included 152 participants at baseline and 145 participants at week 12. Of the 19 participants force-randomized into the no-alcohol content group, 17 completed their week 12 assessment. See Fig 2 for details. The trial ended when at least 30 participants were enrolled in each group at baseline to ensure adequate power to estimate stable effect sizes which stabilize at approximately 30 participants per group [20].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. CONSORT 2010 flow diagram.

https://doi.org/10.1371/journal.pone.0167900.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Participant flow.

https://doi.org/10.1371/journal.pone.0167900.g002

Study Interventions

Participants in all conditions were given the NIAAA guidelines for safe drinking and asked to follow these guidelines to ensure that all participants were aware of the safe drinking guidelines and to offer more credibility to the mobile assessment control, as have been used in previous web-based interventions for alcohol use problems [27]. Participants were led to understand that their messaging program would differ from the others with regard to content and frequency.

Data Analysis

As in previous drinking moderation studies [1], the two primary outcomes were weekly sum of standard drinks (SSD) and weekly sum of heavy drinking days (HDD). These were selected a priori because NIAAA safe drinking guidelines include measures of drinking quantity and intensity. Secondary outcome measures included number of days without drinking per week and consequences of heavy drinking, as measured by the SIP. Both intent-to-treat and completer analyses were performed. To determine whether differential attrition occurred by condition, we investigated whether group differences existed for baseline scores and rates of follow-up at each assessment point. Results indicated no significant differences in attrition between conditions and no significant baseline differences in socio-demographic and alcohol severity variables between those lost to follow-up and those who completed their 12-week program. To analyze the three count outcomes (number of drinks per week, number of heavy drinking days per week, number of days per week per week without drinking), we used Poisson regression models including a scaling parameter (ϕ) to allow for overdispersion and for non-normal distribution of count data [30]. For the continuous outcome (consequences), we conducted ordinary least squares regressions. In all these regression models, we first conducted regressions controlling for baseline values comparing TA to MA on all primary outcome variables to test our primary hypothesis that TA would be superior to MA on all drinking outcomes. We further conducted regression analyses for all outcomes by group, with four dummy-coded indicators comparing each of the four intervention groups with the control group (1 = intervention group, 0 = control group), adjusting for baseline centered at the grand mean. The sample size enabled a test of the secondary hypotheses of a medium effect size at power = .80 when comparing differences between the individual active treatments and the control. We conducted all analyses using IBM SPSS 23 with a significance level set to .05.

Demographics and Mobile Phone Characteristics

Descriptive data for the sample are presented in Table 1. Participants had a mean age of 43.2 (SD = 11.1), and 74.9% were female. The sample was racially and ethnically homogeneous, with 93.5% identifying as white and only 2.4% reporting Hispanic/Latino ethnicity. One hundred and thirty-five participants (79.9%) had owned only one personal cell phone in the last year, and 95.9% reported that they had never had their service turned off for any reason. With regard to text messaging plans, 90% had unlimited SMS and 9.4% a monthly limit. Overall, most participants (80%) sent and received between 0–100 messages per month while 16.5% received between 101–500 messages per month. At baseline, participants were drinking 24.9 (8.4) standard drinks (SDs) per week on average, drinking on 5.2 (1.2) days per week on average, and drinking four or more drinks in a single sitting on 3.4 (2.3) days per week. Participants’ drinking goals skewed towards wanting to reduce both the amount of alcohol consumed per day and number of days drinking per week (78.3%), with 5.1% wanting to reduce the number of days they drank only and another 5.1% wanting to reduce the amount of alcohol consumed per day only. Of those who did not specify any of these goals, 9.4% reported wanting to reduce in general. There were no significant differences between groups at baseline on primary outcome variables. Participants responded to 89.1% of weekly self-tracking, ranging from a low of 74% in the LF group to a high of 97% in the TS group.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Demographics.

https://doi.org/10.1371/journal.pone.0167900.t001

Weekly Sum of Standard Drinks (SSD)

Please see Table 2 for descriptive statistics on all primary outcomes at baseline and week 12, and Table 3 for results of the regression models. Weekly sum of standard drinks (SDD) in the 30 days prior to the week 12 assessment revealed that participants in all treatment groups reduced their weekly alcohol consumption more than participants in the control group except GF (statistical trend, p < .09.), with the TA group yielding the largest effects, b = -0.40, p < .001. Multiple comparisons between treatment groups indicated that no treatment group emerged as significantly better than others, as indicated by overlapping confidence intervals. Changes from the baseline to week 12 by group are presented in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Baseline and Week 12 Drinking Outcomes.

https://doi.org/10.1371/journal.pone.0167900.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of drinking in the four intervention groups (Loss Frame, Gain Frame, Tailored Statically, Tailored Adaptively) with the control group at Week 12: Estimated means and standard errors from residualized regressions, adjusting for baseline levels.

https://doi.org/10.1371/journal.pone.0167900.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Estimated means and standard deviations of number of drinks per week.

https://doi.org/10.1371/journal.pone.0167900.g003

Heavy Drinking Days

Assessment of the weekly sum of heavy drinking days (HDD) revealed a similar pattern, such that a significant main effect was observed for all treatment groups except GF (p =.15) when compared to MA. As with SSD, the TA group yielded the largest effects on this measure, b = -0.72, p < .0001. As before, multiple comparisons between treatment groups indicated that no treatment group emerged as significantly better than others, as indicated by overlapping confidence intervals.

Days without Drinking

All treatment groups increased the number of days without drinking over the course of the trial, except LF (p = .08) when compared to MA, again with the strongest effect in the TA group, b = 0.78, p = .02. As before, multiple comparisons between treatment groups indicated that no treatment group emerged as significantly better than others, as indicated by overlapping confidence intervals.

Consequences

There were no significant reductions in the type or severity of consequences as measured by the SIP over the course of the trial compared to MA, although there was a statistical trend towards consequence reduction in the TA group, b = -0.17, p = 0.07. Post-hoc analyses revealed that those who reduced their drinking to safe levels had significantly lower SIP scores at week 12 than those who did not, F = 5.16, p = .024.

Subjective Goal Achievement

All groups reported significant subjective improvements in their drinking behavior from baseline to week 12 when compared to MA. The percent of participants in each group who reported at least moderate improvement in drinking over the course of the 12 weeks was as follows: 17.9% MA, 34.6% LF, 41.9% GF, 46.7% ST, 51.6% TA, and 31.3% NA. The partial correlation between subjective overall change and actual change in SDD was significant, r = .49, p < .0001. Finally, 79.6% of all participants assessed at week 12 expressed a desire to continue receiving text messages for an additional 12 weeks. However, only 64.4% of the total sample reached out to the research assistant to select and enroll in a new program.

Descriptive Assessment of Non-Alcohol Messaging Group

Overall, the NA group reduced their SSD by over 5.5 drinks per week as compared to 2.5 in the MA group. When examined separately, the weekly drink reduction was similar to that observed in the GF and LF groups, but the effects were smaller and insignificant for both HDD (.54 day decrease) and days without drinking (.66 day increase), when compared to MA. Dichotomous analyses indicate that 31.3% reported at least moderate improvement in their drinking, which is similar to the LF group (34.6%).

Active Messaging Conditions

This study was not powered to detect significant differences between active groups, but the findings can inform future messaging development for health behaviors. Notably, TA consistently had the largest effect sizes among treatment groups, although effect sizes amongst treatment groups did not significantly differ. All messaging groups yielded meaningful reduction in weekly drinking, which highlights that even simple untailored daily messaging can have a positive impact on alcohol-related behavior. That these automated mobile programs significantly reduced heavy drinking has implications for improving the provision of low burden resources to heavy drinking help-seekers.

The larger effect sizes, albeit not significant, of the TA group compared to others adds support for further study of just-in-time adaptive interventions to enhance behavioral outcomes [31, 32]. However, the mechanisms to explain larger effect sizes of the TA intervention could not be determined in the current study. It could be that the additional messages individuals received on their heaviest drinking nights, the “just-in-time” messaging feature, the addition of the participants’ names in several messages, or the adaptive nature of the intervention to the users’ goal achievement from week to week differentially impacted outcomes. It is also possible that the increased quantity of messages received by the TA group accounted for the difference in outcomes, as it has been posited that the benefit of tailoring may actually be in the increased contact associated with most tailored interventions compared to untailored interventions [18]. More research is needed to dismantle the effects of these interventions and understand the mechanisms of efficacy. Fig 3 highlights the descriptive change from baseline to week 12. While no strong conclusions can be made based on this pilot data, these preliminary results are consistent with the larger literature suggesting the benefits of tailoring and adaptation, and should provide guidance for future studies.

Gain and Loss Framing

This was the first text messaging study to compare gain-framed vs. loss-framed mobile messaging for any behavior. We were surprised to find few differences in outcomes between these groups on any measure. While there were slight differences in the effect sizes of these groups compared to MA, they were not significant. Historically, GF and LF have been differentially used such that LF is used to motivate individuals to avoid future problems and GF to facilitate progress [33]. Future research on messaging for PD should examine moderators of gain- and loss-framing used in the health promotion literature in general, such as motivation style, severity and locus of control [15, 17, 34]. In our work developing an SMS intervention for addiction continuing care, we found that while most participants preferred GF messages, those individuals who preferred LF messages perceived fewer overall benefits of changing and had lower readiness to change scores. Understanding these differences in preferences as individuals navigate the change process is an important undertaking and will be explored in later papers.

Other Outcomes

There were no significant reductions in drinking consequences for any of the treatment groups compared to MA, though the TA group’s consequence reductions trended towards significance. This is similar to the findings of previous studies on brief in-person and web-based interventions which revealed few meaningful reductions in alcohol consequences [27, 35, 36]. It is possible that the reductions in drinking quantity or frequency did not translate into significant reductions in consequences. Those who reduced their drinking to safe levels had significantly lower SIP scores at week 12 than those who did not, suggesting the ability of the measure to discriminate between heavier and lighter drinkers. Future research will be needed to understand this discrepancy. It is also possible that the SIP measure may be more appropriate in abstinence-focused studies and that changes in some consequences may not be readily apparent early in the change process of those pursuing moderation goals. Longer-term follow-up assessments are needed to test this.

The addition of the general motivational messaging program containing no reference to alcohol represents important ground work for understanding the differential impact of content specificity vs. non-specificity on behavior change attempts. Although this group was not randomly assigned and therefore results should be viewed as exploratory, the NA group had smaller effect sizes on most outcomes than those of all other active messaging groups. This finding is in line with recent research suggesting that messaging targeted to the presenting problem was more effective in improving health outcomes than non-targeted messaging [4]; however, this finding warrants further study, as general non behavior-specific messaging may be used to circumvent some of the safety and security regulations that currently limit mHealth treatments. HIPAA regulations require not only secure messaging options, but also creative alternatives that help us integrate messaging into care in a cost effective manner. It is important to note that, like the MA group, it appears the NA group may not be as robust over time as the other groups.

While this treatment development and pilot study provide a rationale and foundation for further study of remote mobile SMS as an intervention for problem drinking, several limitations exist. First, sample sizes for each condition were small, limiting power for analyzing multiple treatment groups. Our sample was predominately white and female, limiting generalizability of findings to more diverse samples. The sample enrolled was similar in terms of gender break-down to the 1,149 total individuals screened on the web, suggesting that SMS may be particularly attractive to women seeking help or information online. This study was also very conservative with regard to who was enrolled compared to open real-world, web-based intervention studies, limiting generalizability to the heterogeneous group of PDs that might seek help on the internet. Expanding the reach of future studies to include a larger subset of treatment-seekers might result in significantly more individuals receiving messaging programs. Similarly, future research with larger samples will be needed to test effects over time before long-term efficacy can be determined.

This study utilized consumption assessment methods that at present are not the gold standard metric when assessing in-person alcohol consumption (e.g. Timeline Follow-Back or objective validation of results). Future research with more resources should compare the reliability and validity of these brief measures when completed remotely with validated measures such as the online Timeline Follow-Back [37] or transdermal alcohol sensing. However, there is a robust web-based intervention literature using similar assessment methods, and the relative change between study conditions provides a useful comparison of messaging impact between groups [27, 38]. Despite these limitations, this study provides further evidence for the utility of SMS as a tool to reduce alcohol consumption remotely in a heavy drinking sample. Future research will need to test effects over time before long-term efficacy can be determined.