About Topics (NLP) and how they work

Employee comments provide a rich source of insight and add context to quantitative responses. Whilst every employee comment is valuable, analyzing them manually has a cost and can become time consuming, especially in larger organizations. Peakon Topics’ analysis uses Natural Language Processing (NLP), a field of machine learning, to analyze employee comments and identify important themes and overall sentiment related to issues that matter to the people in your organization.  Peakon’s bottom-up approach makes it possible to surface the most relevant topics in real-time, in multiple languages, without introducing bias. To learn more about how to review your topics, see Doing a qualitative text and sentiment analysis with Topics.

This article covers:

• About topics
• Understanding how topics work
• Supported languages

About topics

Peakon avoids common pitfalls of text analytics described above and provides companies with actionable topics. The unique benefits of Peakon’s approach are:

• Topics are actionable as they are in a context - an ‘Additional Training’ Topic in relation to the Accomplishment driver may require different action than the same Topic in the Peer Relationship driver.
• Easily used in multiple languages - since there is no need for pre-labelling and comments are passed through an algorithm bottom-up, users can get instant translation on the go as well as filtering by the language the comment was written in.
• Extracts real meaning, not biased by commonly occurring words: Peakon down-prioritizes words that are used throughout all survey answers and questions with no specific meaning. A Topic is only generated if it has meaning in a driver context.
• Provides a Topic extract: Peakon summarizes the Topic by putting together the most representative lines from one or multiple comments to provide you with an overview of the meaning of the Topic.
• Comments are highlighted based on their actionability, so that even with 200,000 comments, leaders can focus on what is most important to their teams.

Understanding how topics work

Text analysis

Peakon's Topics NLP (Natural Language Processing) algorithm has the ability to uncover both general, industry-wide employee experience topics, as well as specific team- and time-dependent topics. All of this, without the need to use pre-defined terms or manually add specific keywords of your own. Peakon summarises a topic by combining the most representative sentences from multiple comments to provide you with a meaningful overview.

The platform calculates the comments' topics once the survey round closes at 1:00 AM UTC on the following day. New topics will generate if both conditions are met:

• There are 2 active core driver questions - this could be one driver from the engagement question set and one driver from the custom question set, for example. Topics are generated from drivers, open-ended questions and values' feedback.
• There are 200 or more comments across all kinds of question sets over the last three months (driver, open-ended, custom driver, or value questions) for one single language. It’s therefore unlikely that people leaders further down the hierarchy will have topics populate, when you consider the survey frequency and the manager’s team size, in comparison to leaders further up the hierarchy, who have access to more survey results and comments.
  Older comments (more than three months old) are also revisited and can contribute to new topics, if they are related, however they are not counted in the process of generating new themes.

Peakon will seek to populate the top 50 topics, and then place these under the respective driver. This means that you could potentially have some drivers without topics despite there being more than 300 comments. 

Sentiment analysis

Typically, sentiment analysis refers to an algorithm ‘reading’ a text comment and assigning a numerical value of probability of it being positive or negative, thus giving it a score (e.g. between -1 and +1, with 0 being neutral). This is done based on how keywords were classified in the past - essentially, the more positive words there are in a sentence (e.g. great, friendly, knowledgeable), the more positive sentiment it will have. However, ideas and opinions are complex and aren’t always black or white.

Peakon bypasses the potential inaccuracies of sentiment analysis algorithm by using the score given by employees themselves (0-10) during the survey. This is a more reliable sentiment score as it comes directly from an employee in relation to a specific driver question which sets a context to the sentiment.
As Peakon asks for feedback after every question, our comments are typically short, simple and focused, and rarely express conflicting sentiment. In this reality, the scale responses are a much more accurate way of capturing the sentiment in almost all cases.

How Peakon provides relevant themes

The challenge that the NLP algorithm overcomes is about finding topics in short text answers, that contain the same words which are used in many contexts. Take two examples of comments in which the word manager is used:

1. In response to one of Peakon's growth questions ("I feel that I'm growing professionally") an employee could comment: "I'm definitely growing in a direction that's more interesting to me, thanks to the development plan that my manager created."
2. In response to an open-ended question ("If you had a magic wand what’s the one thing you would change about Kinetar?") an employee could comment: "I really wish we would provide more line manager training, this would be really helpful."
   In example 1, the word "manager" is used in relation to an employee’s interaction with the manager, whereas in example 2 it is used by a manager asking for more training.

The traditional (simple) approach to aggregating text is to count the number of times noun words are used. Often the counts of words can be represented in a word-cloud that, while visually compelling, gives you little information to act on. The reason for this is that it provides no context. Using examples 1 and 2 we would find that ‘manager’ was important as it is used twice. But being told that ‘manager’ is mentioned often gives no context to act on. We get no interesting actionable information about why ‘manager’ is important.
Because of this, the algorithm is configured to make topics more actionable and interesting:

1. Attach topics to a driver: The most innovative aspect of our algorithm is that it will attach the topics to a driver. This provides a context to the topic. Using the answer from example 1 the algorithm might find "Development plan" as a topic if it is a word used often in relation to growth.
2. Find multi-word topics: In example #1 the answer included the phrasing "development plan". If this is commonly used in many answers, Peakon will highlight the topic as "Development plan" rather than just "Development". From example #2 the topic could be "Line manager training", "Manager training" or "Training" depending on how often the words are used across answers.
3. Remove words from the question phrasing: People have a tendency to repeat the words of the question in the answer. In example 1 the answer includes the word ‘growing’ which was also in the question. It is not very useful to find this word as a topic since we already know that we are asking about growth. Therefore Peakon automatically removes the words from the question phrasing.
4. Down-prioritize commonly occurring words: The algorithm down-prioritizes words that are used throughout all the survey answers with no specific meaning. This means that "Manager" is only found as a topic if it has a specific meaning in a driver context. This provides topics that are specific to the driver, for example, "Office" or "Room" for the environment driver or "Pay-rise" for the driver reward. If we did not down-prioritize commonly occurring words we may find "Manager" as a topic for all drivers because it is a frequently used word.
5. Deliver a topic extract: The algorithm generates a paragraph of the sentences that are most indicative of all the comments that make up a topic – giving you an insight into a broad conversation in a few seconds.
6. The algorithm only analyses the last 3 months of comments: This ensures that your topics will change over time. The system will also use any older comments that are also relevant to the topic, however the topic will depend on comments made in the last 3 months of time.

Supported languages 

Each topic contains comments in one language. The language filter on the topics overview page allows you to see topics generated from comments in a specific language.