I Analyzed 12,833 Tweets to Find How Top SaaS Companies Do Twitter Support

I Analyzed 12,844 Tweets to Find How Top SaaS Companies Do Twitter Support process.st/twitter-support/

2/1/2017

I’ve covered our internal support process and analyzed SaaS support and success before, but when it comes to Twitter support, there’s not much data out there.

From what I’ve learned in this study, I’d say that could be because it’s not something the company wants to do.

Is it a necessary evil, brought about by frustrated customers, or is it companies meeting customers on their level?

When you think about users interacting with companies on Twitter, you probably imagine companies trying their best to calm down users so angry they need to vent their frustrations publicly.

I got the idea for this study when I saw the sheer amount of damage control Outlook have to do on Twitter because of their product, and set out to analyze other software giants and how they do Twitter support.

Since Twitter is entirely open to the public and can’t be edited, I thought it was begging for a study on how SaaS companies support their users on Twitter. That’s for a couple of reasons. One, because no one has published data or written a post like this before. And two, because I enjoy writing studies on SaaS companies, like this and this on pricing pages, and this on landing pages.

Here’s my study into the effectiveness and uses of Twitter support at SaaS companies.

How the Top 20 SaaS Companies do Twitter Support from Benjamin Brandall on SlideShare.

Key findings:

1/15

Key findings: Tweets from users directed at support accounts are generally negative/neutral in sentiment, but a higher percentage of negative tweets from users often correlates to a more active support account.

Objective (e.g. problem-solving) responses make up 15% of tweets to users.

Negative (e.g. apologetic, damage control) responses make up 15% of tweets to users.

Positive (e.g. thankful, helpful) responses make up 23% of tweets to users.

Neutral (e.g. requests for more information) make up 62% of tweets to users.

The most apologetic company is LinkedIn, which also receives the most negative incoming communication and ends up providing the lowest amount of objective assistance out of all analyzed companies.

The least apologetic company is Dropbox, which only apologized 1.8% of the time despite receiving the 2nd highest amount of negative tweets and tweeting more subjective (less helpful) messages to users.

The most helpful company is Constant Contact, which provides the highest percentage of objective (problemsolving) tweets.

The least helpful company is LinkedIn, which tweets both the 2nd highest percentage of negative-subjective (damage control) tweets and the lowest percentage of objective (problem-solving) tweets.

Methodology: This study draws from the Montclare SaaS 250 to find the top 20 SaaS companies and their Twitter support feeds.

For companies without a dedicated Twitter account but also offering support on its general Twitter, I used the general Twitter for data but filtered only direct @mention tweets, which indicated they were offering support directly to a user.

If the company only broadcast from their account and never talked directly to user I counted them as not having a support Twitter account.

Using the t command line tool, I exported 1,000 incoming and outgoing tweets per company (limited by some accounts which had tweeted less than the total times) and loaded the data into Google Sheets.

Using Aylien’s Text API, I did sentiment analysis on all 12,833 tweets, which consists of calculating polarity (positive, negative, or neutral sentiment) and objectivity. I loaded the data into Airtable because it’s easy to manipulate data, and I needed views to divide tweets by sentiment, objectivity, and a mixture of the two.

To display the most frequently used words and phrases, I used Online-Utility.org’s Text Analyzer, which is a free online tool.

Summary of the top 20 SaaS companies and their Twitter support presence Here’s a quick breakdown of the top 20 SaaS companies, and their support activity on Twitter:

2/15

(Zoom in) Here’s what you can learn from this data: 13 out of the top 20 SaaS companies use Twitter as a support channel.

If a company offers support on Twitter, they’ve been active for around 5 years.

During these 5 years, the average SaaS company analyzed tweeted 36,463 times to customers.

As a general rule, pure support accounts don’t follow many users. The average is 3,826.

Support accounts have an average of 18,290 followers.

On average, support accounts tweet 16 times per day. The two most active accounts — LinkedIn and Adobe — tweet 59 and 65 times per day respectively.

Since only 65% of the top 20 SaaS companies offer Twitter support , is it fair to say that it’s more of a luxury than a necessity? What do the companies that use Twitter for support really use it for?

What can the prevalence of Twitter support tell us about how and why it’s used?

For this study, I analyzed both incoming and outgoing tweets, meaning the data set includes tweets at the company, and tweets from the company.

I found that the sentiment of incoming tweets was, overall, negative/neutral:

3/15

What can we theorize the purpose of Twitter as a support channel is?

1. A way to support users who prefer Twitter over email or researching the company’s usual support channels 2. A way to have a dedicated account to respond to tweets from users that require support, and direct those users away from the main account 3. A method of damage control when dealing with frustrated users Judging by the percentage of negative incoming tweets to the analyzed SaaS companies, it’s sensible to conclude that damage control plays a major role. A high percentage of neutral tweets indicates that questions (without positive or negative sentiment) are often tweeted at support accounts.

Essentially, the data says that users use Twitter to complain, and to ask questions.

Let’s look at the breakdown of how companies use their Twitter support channels, and what that means.

4/15

The meaning of positive, negative, and neutral sentiment in Twitter support When analyzing the outgoing tweets, I looked at two main data points: subjectivity, and sentiment.

Sentiment is the tone or attitude of the tweet , and is calculated by Aylien. Aylien categorizes the words used and works out the context to determine whether the tweet is positive, negative, or neutral.

Negative sentiment Outgoing tweets with negative sentiment are apologies and requests to take the conversation to direct messages or email. They are mostly responses to frustrated customers.

The more negative sentiment responses a company makes, the more ‘damage control’ they’re doing.

The most common words and phrases used in negative tweets to users are:

5/15

To summarize the meaning of negative sentiment: To apologize to the user To get the user to take their issues away from the public forum of Twitter, and into private messages

Positive sentiment Outgoing tweets with positive sentiment are thankful. They are mostly responses to acknowledge that a customer’s problem has been solved.

The more positive sentiment responses a company makes, the more likely they are to be solving user’s problems directly on Twitter instead of pushing the user to direct messages/email.

The most common words used in positive tweets to users are:

6/15

To summarize the meaning of positive sentiment: To thank the user for their feedback To thank the user for their question / bug submission To acknowledge that the user’s problem has been solved

Neutral sentiment Outgoing tweets with neutral sentiment are often requests. They are usually direct responses to a customer initially reporting an issue.

Neutral sentiment is often indicative of a request for more information, but without an apology or thanks.

The most common words in neutral tweets are:

7/15

To summarize the meaning of neutral tweets: Requests to direct message Requests for more information No apologies or thanks

Dissecting the balance between subjectivity and sentiment Since the data’s available and offers a deeper insight into the way SaaS companies support users through Twitter, it’s helpful to check tweets against combinations of subjectivity and sentiment.

An objective tweet is often a helpful one, giving the user instructions on how to solve their issue.

A subjective tweet is often an apology, acknowledgement, request for more information, or the company thanking the user.

8/15

Going deeper than just pure objectivity or subjectivity, the most useful categories are:

Subjective/Positive: an acknowledgement of the problem being solved

From the data set I analyzed, I concluded that it is fair to draw a correlation between the amount of subjective/positive tweets and the success rate of that company’s support efforts over Twitter. That’s because they’re usually indicative of the company rounding up the conversation and expressing their happiness to help.

Here are two example tweets with subjective/positive scores:

9/15

Subjective/Negative: delivering bad news, or trying to quell an angry customer

Tweets with a subjective/negative score are mostly indicative of a customer-company interaction that didn’t go well.

Either the company is trying to quell an angry customer, or telling them that there isn’t a solution to their problem.

Here are two example tweets with subjective/negative scores:

Objective: pure problem solving Objective language indicates the company is sharing facts and instructions with the user. A high ratio of objective:subjective tweets to users usually shows the company is good at solving problems without taking the issue to direct messages or email.

Here are two objective scoring tweets:

Is it possible to determine the quality of Twitter support with text analysis?

As I mentioned before, sentiment and objectivity is a strong indicator of the quality of support a company offers.

The strongest indicator of a successful support interaction is an objective response. While no company analyzed offers an overwhelmingly high percentage of objective responses, the most problem solving-focused companies are: Constant Contact ServiceNow Salesforce

10/15

Salesforce And the companies which indicate they’ve solved the user’s problem most frequently (with subjective-positive tweets) are: However, it’s not possible to determine which companies are the most effective at support on Twitter yet because as we’ll see, subjective-positive tweets correlate less strongly with support success and more with the incoming negativity from users.

Which companies receive the angriest tweets? How does this affect their responses?

Referring back to the graph illustrating user tweet sentiments…

You’ll see that the companies which receive the most negative feedback from users are: DocuSign (88% negative incoming) LinkedIn (58% negative incoming) Marketo (53% negative incoming) Incidentally, these companies also score highly when measured against the amount of subjective-negative tweets tweeted: DocuSign (20% negative-subjective outgoing) LinkedIn (21% negative-subjective outgoing) Marketo (18% negative-subjective outgoing) As you’d expect, this illustrates that the companies with the most complaints incoming are more likely to spend time on their Twitter support channels apologizing and dealing with frustrated customers than they are actually solving problems.

11/15

The graph below plots the correlation between objective (helpful) tweets against subjective-negative (apologetic) tweets.

The companies with the highest ratio of objective:subjective-negative tweets are, in general, the most successful at helping users over Twitter:

Again, it shows that Constant Contact, ServiceNow and Salesforce have relatively few pure apologetic tweets compared to their tweets where they objectively help users solve problems.

Which companies are the most and least apologetic over Twitter?

In addition to analyzing sentiment to work out which tweets were written to calm a frustrated customer, I also looked at the number of definite apologies in the text by searching for the words ‘sorry’, ‘apologize’, ‘apologise’, ‘apologies’, etc.

12/15

What can this data tell us?

Two out of three of the top three companies by apology are also the companies which receive the most negative tweets: LinkedIn and Marketo. It’s safe to assume their high percentage of apologies is directly related to the amount of things they should be apologizing for.

Constant Contact, however, apologize at a much higher rate than they receive negative tweets showing they’re more keen to show empathy and assume responsibility for their user’s issues than other companies.

It’s particularly strange and intriguing that ServiceNow and Quickbooks didn’t offer a single apology. While ServiceNow receives comparatively little negativity from its users on Twitter, the same can’t be said for Quickbooks:

13/15

The best and worst SaaS companies at Twitter support From this data set, I’ve found that the key indicators of helpful and empathetic support over Twitter are a high percentage of objective tweets, a high percentage of subjective-positive tweets, and a low percentage of subjective-negative.

With that, let’s aggregate the most and least effective companies at offering support over Twitter:

Worst: DocuSign, LinkedIn, and Marketo To start off with, the poor quality of support from these companies is reflected in the negative tweets they receive.

DocuSign, LinkedIn, and Marketo are the three biggest recipients of negative tweets from users. They are also amongst the top ranked for subjective-negative (damage control, apologetic) tweets. LinkedIn’s actual assistance over Twitter (objective tweet percentage) is the lowest of all companies analyzed, and Marketo also ranks poorly for that factor.

Best: ServiceNow and Constant Contact As the two companies with the highest percentages of objective tweets, it’s safe to say that ServiceNow and Constant Contact are the most helpful companies over Twitter. This is supported by the fact that they receive the 2nd and 3rd lowest amounts of negative tweets from users. While Constant Contact does have a relatively high amount of tweets with a subjective-negative score, you can also see that they score highly for apologies, which isn’t necessarily a bad thing, and, given the good scores in other areas, could be chalked up to empathy.

What can we learn from this data?

It’s obvious that the most effective problem solvers — the ones that tweet more objectively — offer solutions to users on Twitter, not over direct messages or email. They are also the least likely to get negative tweets from users overall.

14/15

From this, you could conclude that users go to Twitter because it’s their preferred method of communication, and if companies are able to solve issues on Twitter instead of pushing the user to emails or direct messages, users are happier and the sentiment of incoming tweets is also impacted positively.

15/15