LAS VEGAS — The cyberenvironment is complex, unpredictable, high risk and comes with real stress, and the National Security Agency wanted to know just how impactful it was on its operators.
In efforts to focus on the health and safety of its workforce, NSA created a survey to assess cognitive stress in its actual tactical cyberoperations. What the agency found was interesting: While their missions do cause stress, operators don’t let it impact how they assess their performance.
But what does affect this performance is frustration, according to Celeste Paul, senior researcher in the Computer & Analytic Sciences Research group at NSA. She and Josiah Dykstra, computer security researcher at NSA, spoke at Black Hat Aug. 8 about their work on cyberoperators’ stress. NSA has various types of defensive operations, such as its red team of authorized hackers that emulate known attackers to find Defense Department network vulnerabilities, and operators using intelligence to proactively look for adversaries in NSA’s own networks.
These are all hands-on tactical cyberoperations. Performance in this line of work relies on speed, precision and a highly trained workforce. Risk comes with the mission, and failure has very real consequences. These pressures are strong, and the stress on these operators can be harmful.
Cyberoperations Stress Survey
NSA is particularly concerned about three stress factors: fatigue, frustration and cognitive workload (the amount of mental effort needed to use working memory).
To study fatigue, NSA used the Samn-Perelli Fatigue Scale, originally aimed at measuring fatigue in pilots. To study frustration and cognitive workload, the agency used NASA’s Task Load Index, which rates perceived workloads to assess a mission or task.
The cognitive work assessed mental, physical and time demand, self-assessed performance, frustration, and effort, which asked how hard the operator worked to accomplish a mission. The questionnaire was distributed to operators at the beginning and end of a mission to capture the entire experience and differences, Paul said.
A total of 126 civilian and military tactical cyberoperators in four NSA locations received the survey. There were 361 operation surveys, and the operators had the same training and worked on the same types of operations, all around five hours long.
What Did NSA Find?
It should come as no surprise, Paul said, that tactical cyberoperations cause stress. Fatigue increased 16 percent and frustration 12 percent.
All the factors measured under cognitive workload were related or linked in some way, meaning they all go up or down together. However, performance was the one factor not linked to the rest. No matter how these factors shifted, there was no relationship to the self-assessment of performance.
Performance wasn’t even tied to fatigue. Even for operations more or less tiring than others, self-assessment of performance wasn’t affected. But NSA did find a correlation between frustration and performance. As frustration spikes, self-assessment of performance plummets.
“The only thing that’s really affecting how well people think that they’re doing in their operations is how frustrated they are with the operation,” Paul said.
So, tactical cyberoperations increase fatigue, frustration and cognitive work, and longer operations are more tiring, frustrating and mentally demanding. Yet, operators always pull through with performance, “but at what cost?” Dykstra asked.
So, What Does This Mean?
The operators are “highly skilled, highly trained, very dedicated to the mission . . . they have pride in their work, always want to win,” Paul said, and she thinks this feeds this weird relationship between performance and frustration.
“They put the success of the mission above all else, even themselves,” she added. “They're taking on this mental demand, physical demand, time pressure and effort, and that's just part of the environment. But the frustration is something that they can’t take on.”
Paul is concerned this frustration can lead to chronic and episodic stress conditions. And while they’re taking on this national security risk, “we, also, as part of the community, have to make sure that they are taking care of themselves,” Paul said.
What’s NSA Doing?
Paul said NSA is making progress in helping address these concerns, and the agency will make this survey public.
Dykstra also outlined how this survey can apply to other operational environments, including using it to evaluate internal operations and measure the differences and changes in a workplace, reviewing policy around task length, breaks and scheduling, checking up on the mental and physical health of operators, remembering is it helpful for people to have some control over their environments as it lowers stress, and empowering operators with a happy, healthy work environment.
This survey can also be helpful to understand decisions for training and recruiting, as it’s helped NSA.