Want to design policies that really work? Test them on the users who need them first
A step-by-step guide to how New Jersey used plain language and user-testing to improve the state’s paid family and medical leave program
Blog Post
March 31, 2021
Paid family and medical leave policies have the potential to promote health and wellbeing, reduce infant mortality, help family economic stability, improve worker productivity and morale and, research shows, can even lead to longer life. But not if people don’t use it. And in the handful of states that have passed such policies, the data shows that many who are eligible and who need it, aren’t using paid leave.
Understanding why – and what to do about it – is what the Better Life Lab and New Practice Lab at New America set out to find in a unique, four-week discovery sprint in 2019, in partnership with the state of New Jersey. One of the key learnings that emerged is also a key part of the answer: you need to think like the user does. And the best way to do that is to ask them.
The first thing to understand that can shape a user mindset is that paid family and medical leave is a rarity in the United States. The U.S. is the only wealthy country that does not guarantee workers paid family and medical leave, and about 80 percent of the workforce is not covered through an employer’s voluntary plan. New Jersey was one of the first and remains one of only a handful of states to offer paid family and medical leave.
So many workers in New Jersey – and other states with paid leave – simply don’t know about the policy. They figure their employers would let them know about it, which many don’t. And some workers, said Yarrow Willman-Cole, the workplace justice program director for New Jersey Citizen Action, don’t think they’re eligible or that they deserve it. “They just assume it’s not for them,” she said.
Paid leave laws can be confusing. Applying for it complicated
The next thing to understand from the user’s point of view is that applying for paid leave is not easy. In New Jersey, potential users have to wade through an alphabet soup of four different state and federal laws to ensure they get the benefits they’re entitled to and that their job is protected while on leave. The process is complicated enough – and the information conveyed confusing enough – that many workers may be dissuaded from applying in the first place, then they tell friends and coworkers it’s not worth the effort. And word of mouth gets around.
“It’s … confusing,” said Jenni Seeds, a disability claims examiner for New Jersey’s Department of Labor and Workforce Development. Paid leave for pregnant mothers and birthing parents, for example, is covered under one of the four laws – three state and one federal – that make up its paid family and medical leave program: Temporary Disability Insurance (TDI). TDI also covers workers who need to care for their own illness or injury, like recovering from surgery or cancer treatments. Family Leave Insurance, (FLI) a separate state law, covers bonding leave for new parents as well as caregiving leave for workers who need time to care for a loved one. [1] “Many people don’t think of their pregnancy as a disability,” Seeds said. “So a lot of times they don’t think that’s what they should be applying for.”
Pregnant and birthing parents are typically eligible for 10 to 12 weeks of paid TDI leave, though they could claim up to 26 weeks if complications arise. [2] They can then apply for and receive another 12 weeks of paid FLI to bond with their new child. Fathers and non-birthing parents don’t qualify for TDI, but, like birthing parents, they can qualify for the 12-week FLI bonding leave. Neither TDI nor FLI ensure that a worker’s job will be held open for them while they’re on leave. But workers’ jobs are typically protected under the state’s New Jersey Family Leave Act, which is enforced by a different agency, or the federal Family and Medical Leave Act, though each have different eligibility requirements.
Still with me?
The process is so confusing that advocates like Williams-Cole have made their own diagrams, with arrows and color coding, and posted it on Facebook. Others have made informal “cheat sheets” to try to help people wade through the alphabet soup of acronyms to get the benefits that they already pay into through a payroll tax. (Employers and employees contribute to TDI. Only employees contribute to FLI.)
Even after updates to the state’s website and brochures trying to explain the program clearly, “people had a lot of questions,” said Holly Low, manager of strategic outreach for the state Department of Labor. “I don’t think we were communicating as well as we could have,” Seeds added.
User Interviews Reveal Pain Points
So, during the sprint, which focused on paid maternity leave for birthing parents, Seeds and Low worked with Lindsey Wagner, a user experience (UX) designer and researcher on the sprint team, on a series of user testing to better understand what users needed to know and where they were getting stuck.
Wagner and the other researchers on the sprint team began with extensive qualitative interviews with 15 potential users of the TDI and FLI programs. Pretty quickly, some common pain points emerged.
“As I started to code the interviews, I started to see patterns to problems,” Wagner said. Users were worried that if they took leave their jobs wouldn’t be protected – something the state’s brochures and website weren’t very clear about.
“Job protection was an immediate initial barrier,” Wagner continued. “It was one of the first questions asked. If you got the wrong information, it could really change your ability to use this program that you’ve already paid into. Paid leave exists for people to use,” Wagner continued. “But some wouldn’t get a chance to use it because they had been led down the wrong path, information-wise.”
At the time, New Jersey’s state DOL brochures and website didn’t have much information on job protection because the laws are enforced through the state’s Office of the Attorney General’s Division on Civil Rights, not DOL. NJDOL officials didn’t want to speak for another agency or run the risk of conveying the information incorrectly.
“Through this process, we really learned that saying nothing about job protection, or avoiding the issue wasn’t helping,” Seeds said. “Even if we’re not the experts on job protection, we realized we have to give people something to at least direct them to the right place to find more information. I think, for me, that was the biggest lesson.”
Added Wagner, “I understand the hesitancy around not wanting to misspeak. But, I explained that, to the end user, you’re all ‘the government’ to them. They don’t necessarily understand which department is in charge of what.”
The interviews also made clear that all the acronyms of the four different laws was simply a complicated blur to most users and that they couldn’t count on their employers to help them sort it all out. Users really needed clear and easy to understand information from the state itself. And the interviews helped state officials understand that many users – men in particular – felt too guilty to apply for and take all the time they were eligible for.
User Testing Leads to Clarity
Online video chats where potential users reacted to brochure material helped sprint researchers and New Jersey officials see that it was much easier and more intuitive for users to relate to life events than to a program acronym. In one such session designed to test how the state conveyed information, the user confusion was painfully apparent. When one user was asked how she would know if she earned enough to qualify for paid family leave, she hesitated, then replied, “I’m not sure, to be honest.”
So Wagner, Seeds and Low worked together to draft a prototype of a new brochure and test it with potential users. “Brochures are particularly difficult,” Wagner said. “You have to be concise, but you also have to be fairly comprehensive. This might be your only touch point with a user.”
The prototypes sought to build from the sprint learnings about where users got stuck, what they most needed to know, and how information could be more clearly communicated from their point of view. The first prototype was therefore designed by starting with the life experience that the user may be going through, and then lead them to the correct state program, not the other way around.
Brochure Prototype 1
After each user interview, Wagner, Seeds and Low met and discussed what seemed to be working and where users were still confused. They would then update the prototype based on what they saw and heard and test that with the next user.
For instance, after testing the first prototype with one user, the sprint team discovered that the user thought she could take six weeks of bonding leave all at once, followed by another six weeks of bonding leave broken into parts, for 12 weeks total. “Clearly, we didn’t want to convey that,” Seeds said. “That was a huge lesson for me, because I didn’t even think of that.”
Users were also confused about how taking company paid sick leave or paid vacation could impact their state-covered paid leave time. “The wording was not resonating with the first few testers, “ said Seeds, “So we tweaked it.”
The group revised the prototype and continued testing.
Brochure Prototype 2
In July, 2020, New Jersey doubled the length of paid leave for both TDI and FLI from six to 12 weeks for each program and increased the cap on weekly pay. In August, Seeds and Low worked together to do user testing with community health workers and used their input to revise the brochures and information with the updated provisions.
Brochure Prototype 3 (Post-Sprint Testing in August 2020)
During the 2019 sprint, knowing that job protection was a key sticking point for users, Seeds, Low and Wagner also tested messages around job protection with users.
Job Protection Prototype 1
Job Protection Prototype 2
Based on user testing, Seeds said, the updated version included the words “job protection” more explicitly, rather than just “protection,” and, rather than use graphic images, defined the target audience for each benefit – either those taking pregnancy and delivery leave, or TDI, or those taking bonding or caregiving leave, or FLI.
“It was cool to see it progress as each person viewed it,” Seeds said. “It got a little better.”
Added Low, “I remember thinking, ‘My goal is to write clear, understandable things, and you just need regular people to look at things and tell you what they really think. Sometimes we get bogged down in the weeds, and it all makes sense in our heads. You really need that reality check.”
Work In Progress – with the user in mind
After the sprint, Low and Seeds have continued to work to incorporate user experience and user testing in developing new brochures and materials that are easily understandable and intuitive. They worked with New Jersey’s Division on Civil Rights in order to incorporate more information on job protection. And they’ve been inspired by the diagrams that advocates have used to communicate with potential users, such as those designed by the NJ Time to Care Coalition.
Time to Care Coalition Diagram
Source: NJ Time to Care Coalition
In March 2021, the state released an updated brochure that was inspired by learnings from the sprint and the NJ Time to Care Coalition diagram and designed with users in mind. “The biggest change with the final updated brochure is that, based on user testing, we realized we needed one graphic that shows the two benefit programs and the two job protection laws all together, to try to illustrate the connections between them, because this is where we kept losing people during testing,” Low said. “We'd show users a slide on benefits and then a slide on job protection (shown above), and they still weren't seeing how all four connected.”
In addition to the new brochures, Seeds and Low said the state has worked to update their website with users in mind and has installed a chatbot to help people navigate through the application process. Based on some of the sprint findings, NJDOL also ran a social media ad campaign on Instagram, Twitter and Facebook targeting fathers, particularly fathers of color, in an effort to normalize their taking paid family leave. They worked with the state Division on Civil Rights to incorporate wording on the job protection law too. Because user testing showed a clear pain point – that employees often get the wrong or misleading information from their employers or HR departments – NJDOL has also put together an Employer Toolkit to better educate employers on the paid family leave program, so they, in turn, can better educate workers. “We’ve been tweaking it as we go,” Low said. “Our next step with that is to try to do some user testing with employers.”
Seeds said NJDOL is still gathering data but that, anecdotally, she is seeing less confusion. Several years ago, she said people were frequently applying for the wrong program. “They saw the word ‘family” and thought ‘That’s what applies to me,” and skipped disability entirely,” she said. “I’ve noticed a large decrease in that just with answering emails and hearing about the complaints in the call center.
“I can’t say everything is solved,” Seeds continued, “But it’s an improvement and a work in progress. And it’s helped people a lot.”