Though the team at Cision Insights relies on state-of-the art technologies and tools to solve problems for some of the most successful companies and institutions around the world, it’s the people behind the scenes—and their ability to develop and execute customized research that produces actionable intelligence that sets it apart.
Data on its own is valuable. Data combined with human qualitative research and in-depth analysis from industry experts is everything.
In this interview, Karen Stockert, one of our brilliant analysts who specializes in the institutional and nonprofit space at Cision Insights, discusses the impactful work she delivers for clients and she applies communications design to research and analysis.
Cision: Please introduce yourself and tell us about your role at Cision Insights.
Karen Stockert: I'm a client insights manager in the health and nonprofit sector, and I've been with Cision for about 10 years. I started as a social media research analyst with a company called Visible Technologies, which was later acquired by Cision.
And could you talk to us a little bit about your path from starting out in communications design to becoming a media analyst at Cision Insights?
Years into my career in visual communications design, I started to recognize that visual communicators and myself included, weren't always as effective as we could be at understanding and getting our messages through to our audiences.
This led me to pursue my master’s in intercultural relations. My degree program combined aspects of anthropology, which is the study of cultures and the study of communications. And that’s when I sort of started to recognize, wow, research is really interesting to me.
I also discovered about that time that some forward-thinking companies, Intel, as an example, were hiring anthropologists to understand their global consumers so they better design products that address the needs of their ever more diverse consumer base. There was a recognition that the developer or designer doesn't have all the answers and that an ethnographic researcher can help bridge that gap.
So what exactly does an ethnographic researcher do?
As an ethnographic researcher, my team and I would get consumers out of the focus group. We would interview them in their homes, as they shopped, sometimes with their friend group over coffee or happy hour, something like that.
These settings and the interactions that we observed were effective in surfacing a deeper understanding of consumer behaviors and attitudes. Particularly, those behaviors that people couldn't always articulate when we simply asked [survey] questions. So from the research and analysis skills that I developed, it was a short hop to apply to consumer insights in social media.
Among your recent work at Cision Insights is a collaboration with the Alliance for Science to study the spread of COVID vaccine misinformation, its prevalence in mainstream news media, and the potential for it to influence public perception. What was the client's initial question or the problem that they came to Cision with, to be addressed?
A big focus for the Alliance for Science and the work we'd already been doing with them was about countering conspiracy theories and misinformation in what are often contentious science issues, genetic engineering, for example, or in this case, vaccines. We'd already helped them with a study on Covid misinformation shortly after the pandemic took hold and that was quite successful. So Covid vaccine misinformation in mainstream media was the next logical area of investigation and we were eager to help them with it.
In this project as the Cision Insights analyst, how did you go about setting up methodologies to execute the research?
For the misinformation studies, our methodology was an evolving process because we hadn't done anything quite like this before, but we honed our process from the first report first study on COVID misinformation. So our methodology developed with each successive study.
We used Cision's Next-Gen Comms Cloud platform. And it was really useful because it gave us access to just millions of English-language online news sites and news-focused blogs and historical content. We had a lot at our fingertips, so our challenge really was in wrangling this vast amount of content to derive meaningful insights that answer our client's questions.
"We had a lot at our fingertips, so our challenge really was in wrangling this vast amount of content to derive meaningful insights that answer our client's questions."
We started with search queries for misinformation themes within Covid vaccine coverage that we refined repeatedly until they returned relevant results. Other search terms were added on to identify content that referenced or promoted misinformation, with or without fact checking. We further narrowed our data set by limiting the outlets we tracked to the 100 highest reaching English-language global news outlets and blogs.
Another tool that we use is human coding. An actual person reads and codes so that we can have a higher level of confidence in what is actually being said in the misinformation content. In this case we randomly selected 500 articles that were then coded for misinformation themes and whether those was being fact checked or not.
We documented our methodology thoroughly, especially in this case, because we knew the client planned to use the results of our study for an article to be submitted to a peer-reviewed publication. And in those cases, it's very important that our methods are bulletproof. We were able to test the completeness and accuracy of our search across multiple external sources.
How do you go about bulletproofing one's methodology in a significant study like that?
[We set up] a sensitivity test, which essentially helped us determine whether the searches that we had set up were actually bringing in the relevant content only. And that gave us that additional level of confidence in our data collection.
How were the measurement methodologies for this research innovative?
One of the things about our measurement that was innovative and contributed to our insights in new ways was the taxonomy of misinformation types, a grading of how overt the misinformation claims and whether fact-checking was employed. This determination was part of the coding process in our human-coded sample of articles.
Another novel aspect of our methodology involved determining how to narrow the data set for such a large topic: Covid vaccines. We decided to focus our analysis on coverage in just the 100 highest-reaching English-language online news outlets. To identify these, we sampled articles at regular intervals throughout 2020 and combined those into a single list to surface the 100 top-reaching outlets. Our data set comprised articles from those outlets only.
With Cision being a leader in measuring and quantifying narratives for corporations, nonprofits, and brands, there must be a multitude of data sets at your disposal. How many data sources were tapped for that particular engagement? And how did you determine which would be the most effective in finding an answer to the question at hand?
We have a number of tools at our disposal. Much of our regular reporting for this particular client [is held in a proprietary system]. The Insights media suite is great because there are dashboards set up that allow the client to interact with their content and do their own searches and monitoring.
Beyond this particular project at Cision Insights, it sounds like identifying different data sets, fine-tuning queries, and then actual capture and measuring is critical in the work that you do. When it comes to media analysis, how much of it is from pure data and how much comes from humans and human intelligence?
In the work we do, the human element is the differentiator. That's really our value add. [I had a] similar conversation very recently about how soon artificial intelligence or AI might put the insights analysts out of work.
"In the work we do, the human element is the differentiator. That's really our value add."
And I think right now, it's looking like there's a lot that AI can offer in terms of helping crunch huge amounts of data or identifying patterns, but it's still not able to reliably provide that discernment or to make the big picture connections that an experienced analyst can bring to the questions our clients care about.
It seems that AI still has a long way to go and a lot to learn.
I think so, but it's changing fast, so hold on tight.
What are some other projects you’ve led at Cision Insights?
One of the things I like most about the work we do in the nonprofit sector is helping our clients understand how to be more effective at communicating around some of the most pressing issues of the day.
For example, an environmental advocacy client asked us to track the groundbreaking work they are doing to raise global awareness of climate-warming methane emissions and the impact mitigation can have on slowing the rate of climate change.
One of the human rights organizations we work with asked us to help them track the ongoing relevancy of their own in-depth reports on mental health staffing in schools, marijuana reform, immigration policy and more by measuring to what extent their reports are referenced or quoted in articles and posts on these issues over time.
"One of the things I like most about the work we do in the nonprofit sector is helping our clients understand how to be more effective at communicating around some of the most pressing issues of the day."
One more question. What are a few resources you'd recommend to people interested in learning more about data and analysis in media?
As far as being a better data analyst or working with data in the most effective way, I think a great resource for people to check out if they're not already aware of it, is a website called Vizlogue. Rebeca Pop is a data visualization rockstar. She does some really beautiful and interesting things with charts, and it's just cool to see somebody who's so interested in making data more visually compelling and interesting, just using different ways of visualizing data.
From my graphic design days, I'm a big fan of Edward Tufte, whose books include Beautiful Evidence and The Visual Display of Quantitative Information. Those are some classics.
I also kind of keep an eye on LinkedIn. There’s always things going on there that are related to our industry. On The Media is a fascinating podcast for understanding how the media portrays what’s going on in the world.
This interview has been edited and condensed for clarity.
