Jon Cairns

TL;DR

If you use sferik's Twitter gem with multiple accounts in a rails app, or a ruby app that uses concurrency in some way, then you need to be using the thread safe configuration.

The vicious bug

This was one of the most difficult bugs to diagnose that I can remember. I work on a large rails application that collates data from various social networks, for multiple accounts. Each application account can have one or more Twitter accounts linked to it, and data is retrieved from the Twitter streams multiple times a day.

One day, we noticed that the Twitter data was crossing over between accounts in our production environment. The tweets that should have been in one account were appearing for a totally separate account. What made it worse was that it seemed to be random; the bug only appeared on some occasions, and it worked perfectly on others.

I tried replicating the issue on my development PC, but couldn't get it to occur. As much as I tried to make it fail, the API communication appeared to be working perfectly. I spent hours walking through the code, line-by-line, writing tests to try and reproduce the problem. No luck.

I then took a look at the configuration of sferik's Twitter gem, which we use for communicating with the twitter API:

Twitter.configure do |config|
  config.consumer_key = YOUR_CONSUMER_KEY
  config.consumer_secret = YOUR_CONSUMER_SECRET
  config.oauth_token = YOUR_OAUTH_TOKEN
  config.oauth_token_secret = YOUR_OAUTH_TOKEN_SECRET
end

While looking at this code, it suddenly hit me: the configuration is a class method called on a globally available module. That essentially means that a global variable is hiding behind the scenes.

It then occurred to me what was happening on the application level. The configuration details for one Twitter account were being passed in, and then at some point during the API communication, another thread or process was reconfiguring the global Twitter object with another Twitter account's details. We use Sidekiq to schedule jobs, and it runs jobs in parallel. The issue never appeared on development because we didn't fire up the Sidekiq instance and simulate the whole process; it was easier to test the API communication in isolation, but this stopped the problem from occurring altogether.

The thread safe way

It turned out that the standard way of using the Twitter gem is not thread safe. This is actually addressed in the README, but you have to scroll quite a long way down. Most people would grab the configuration from the Quick Start section, which is probably fine for most people, and is exactly what we did. The thread safe way of configuring a Twitter client is:

client = Twitter::Client.new(
  consumer_key: "an application's consumer key",
  consumer_secret: "an application's consumer secret",
  oauth_token: "a user's access token",
  oauth_token_secret: "a user's access secret"
)

You then keep a hold of the client and make the relevant calls on that rather than the globally available Twitter object.

The problem was one relating to evolution. We didn't start with Sidekiq and concurrent Twitter API retrieval, this grew gradually over time. I thought I'd share it just in case it helps someone else who may be banging their head against a wall at this very moment.