Competitive Research Tools: Data Analysis

This is the first post in a series looking at competitive research tools and their data accuracy. We’ll be looking at today.

dog licks ice cream

Competitive research tools are meant to serve one purpose, to get accurate data on the competition and turn that into actionable items. So how do you know ifthe data that you are getting is actually accurate or not? Perhaps you’ve plugged in one or two of your own accounts, maybe you’ve just trusted the data, or maybe you’ve not cared. Well now is the time to care! Sharing is caring, so we are going to share data on several of clients and compare that to the competitive research tools data.

As of now I have NO IDEA what we will come up with.

First, let’s get a little information about and their features. If you want a full review of their tool Google “spyfu review”, however for this post we are going to focus on accuracy of data. Features:

  • Classic- allows you to see competitors budget and keywords.
  • Kombat- gives you the keyword overlap of up to 3 competitors.
  • Keyword SmartSearch- allows you to generate keyword lists and easily filter.
  • Ad History- shows a month over month history of ads.

Down and Dirty with Data

We’re going to take different accounts and test the following areas for accuracy:

  • Budget
  • Keywords


Spyfu gives a range for the budget. We took the mean of the range to make the data easier to understand. Our numbers were taken from separate dates within the last two weeks.






There is obviously a significant difference in the budgets. This got me thinking that perhaps Spyfu is only tracking budgets on the search network. Here is how budgets look when only looking at numbers within search.






Average percentage difference was still 39% higher, but more accurate. Lastly, I took the average previous months budget and compared Spyfu’s previous months budget (found by hovering over the previous month in the chart).






This was the closest to being accurate. With seasonal and accounts less than 3 months old being most inaccurate.


  • The previous months data, found by hovering over the previous month in the graph, was more accurate than the current months data, which is prominently displayed.
  • Numbers were more accurate for accounts with budgets that were consistent and unchanging for several months.
  • Seasonal clients and accounts with less than 3 months data were significantly off.


For keywords we took the entire list of active keywords in our accounts (no paused, low search volume, etc.), took out any match types (because it appears that Spyfu does not account for match types) and then compared our total number to their total number.







  • It appears that the larger the account the larger the discrepancy. However, in those accounts there are also more long-tail keywords, which may account for that difference. I compared keywords for one account that had received impressions to Spyfu’s keywords and found that there were few in common. One possible conclusion for this is that broad or phrase match keywords were triggering other variations of keywords, relevant of course.
  • Keyword count and accuracy are more inaccurate for larger accounts.
  • Keyword lists are not 100% accurate so they should be not be assumed to be winners immediately(duh?).

While Spyfu’s data is not completely accurate there are still many benefits to using the tool for competitive research. Budgets are slightly off but can still be used to get an idea of where budgets in your industry are at. By comparing several competitors you can tell who is spending the most. The keyword tool can give you keywords to expand your account or get it started. The Kombat tool seems to be the most useful tool that Spyfu has to offer.

I’m interested to hear how accurate is with your own data. What do you find when you plug in your own account(s)?


****Update – March 19 – Response From Spyfu****

Hey Luke,

Mike Roberts with SpyFu. This is cool stuff.

You’re definitely right that our budgets are search only; no content network, no partner sites.

When we publish the budget numbers we post a range. Something like $165-$330. The fewer data points we have the wider that range is; it’s an expression of uncertainty --> very much like presidential polling sampling errors (+-6%). The larger the sample size (inclusive of history), the more accurate the results.

I’m curious what you did to turn our budget into one number. Did you take the bottom number, the top number, the mean? [I took the mean -Luke]

I’d be curious how often your client’s budget falls within the actual range. [I added this to the chart]

In any case, calculating someone’s ad budget from what we see them advertise on is akin to predicting intelligent life on other planets based on the wavelength of their star. There are so many factors that determine budget (Quality score, bid strategy, position, cost per click [which we get from Google, but they don’t report accurately], Shopping, Videos, Images)

Here’s the thing: When we say someone advertises on a keyword, it’s 100% true — there’s a cache page you can use as an audit trail going back to 2006. Ad budget is the biggest extrapolation we make; the farthest from the raw data. My goal when I first calculated that number was to make it so that given that I know my own Adwords budget, can I estimate how much bigger (or smaller) a player my competitors are in the market. Then, secondarily, I wanted to be able to roughly gauge the size of a client, competitor, or partner; are they spending hundreds, thousands, tens of thousands, millions? It’s kind of hard to tell by looking at someone’s website, right? Make sense?

Anyway, I love what you’re doing. Thx.



photo courtesy of

24 thoughts on “Competitive Research Tools: Data Analysis

  1. Cody Baird says:

    Great post Luke. I have been hot and cold on spyfu over the years. Thanks for doing this reasearch and sharing. This confirms why I have seen wide discrepancies at times.

  2. logo items says:

    I think that one of SpyFu’s core strengths remains one of the areas that’s been there since the Classic version, their estimates for competing PPC spend volumes. While I tend to error in favor of KeywordSpy in terms of depth of keywords intelligence, in terms of accuracy on the competitive PPC budgeting considerations I’ve found SpyFu’s estimates tend to be more accurate (I’ve tested on a few instances of my own).

  3. Mike Roberts says:

    Hey Luke,

    Thanks for posting my response; that’s awesome.

    I just re-read the article and noticed a part that I hadn’t before:
    “Numbers were more accurate for accounts with budgets that were consistent and unchanging for several months.

    Seasonal clients and accounts with less than 3 months data were significantly off.”

    It’s really interesting that you notice this. I’m impressed, because this speaks to a subtlety about the system that I didn’t think anybody had ever noticed.

    Sometime last year, we started smoothing our budget estimates using a weighted rolling average.

    So, if we see your domain show up on a keyword this month, but not last month or the month before, we won’t assign all of it’s value to your budget. In fact, you aren’t assigned a keywords full budget *until* you’ve been on it for 3 or more months.

    So, that’s why accounts that are less than 3 months old are inaccurate; we value historical bidding quite a bit, and it takes that much history for our estimates to reach “normal” levels.

    I’m totally impressed that you were able to observe the data and conclude what was going on with so few data points. Very cool.

  4. DEP Ecommerce Consultants says:

    Hey Luke,

    Good stuff. We just dealt with this over the weekend when a client asked for quick competitive audit. We were getting really different numbers from Spyfu and SEMRush. I’d love to see you profile SEMRush next.



  5. Alison Macdonald says:

    Hi Luke
    fantastic study thanks – and actually a great reply from Spyfu too, so double whammy.

    Yeah, we’re always cautious when looking at the results from research tools – often best used as an approximate guideline. It amazes me (or maybe doesn’t) the amount of clients we take on who still take the Google Keywords tool to be the last work in analysis.

  6. Katie Saxon says:

    Great analysis Luke – I have wondered before how much trust to place in the SpyFu data, but I’ve never taken the time to investigate! Really good to know what kind of margin for error we can expect from their stats, so thanks.

  7. lukealley says:

    @Mike Thanks for reading and responding to the post. I included your response so everyone got a full picture of the tool, which we’ve found very useful at DSM. Hopefully with our small (and not scientific) study everyone will have a better understanding of some of the ins and outs of Spyfu. You’re explanations have clarified a lot. Thanks again!

  8. lukealley says:

    @DEP Thanks. That’s coming next!

    @Kyle Much appreciated!

    @Alison Very true. We find Spyfu, or any competitive research tool, adds another angle in doing research. Google KW tool gives us CPC estimates, Spyfu gives us budget estimates. Helps make recommendations.

  9. Meer klanten says:

    I am realy wondering if this is beneficial at all…
    I am working with Adwords without any tools, just handson working the numbers and focusing on that what is happening with the Campaigns and website.

    Will read more about beneficial tools like SpyFu


  10. Henry says:

    Agree that Spyfu is a great SEO tool. It has helped me to better manage many of my campaigns. Your post has given me new insights to explore with regards to client management 🙂

  11. Anonymous says:

    It’s really interesting that you notice this. I’m impressed, because this speaks to a subtlety about the system that I didn’t think anybody had ever noticed.

  12. Peter says:

    Hey Luke,

    Best stuff. We just dealt with this over the weekend when a client asked for quick competitive audit. We were getting really different numbers from Spyfu and SEMRush. I’d love to see you profile SEMRush next.

    I hope it will be have more new best skill

  13. John Ernest says:

    I believe there are online research tools that you can use online but the best of them are paid for. So if you really want to be number one and have a spy on your competition it is advisable that you invest or you can cry in the dust. Thank you for the great post!

  14. Pingback: Competitive Research for SEO and Search Advertising | 

  15. Pingback: How to know what your competitors are doing? Competitive research | 

  16. clguy1234 says:

    Fantastically useful post, Luke. +1 to suggesting a similar review of SEMrush next. (Contact me if you do this – I have a pile of data and some observations that might be helpful)

    While the budget #s are of course important, I would love for you to compare click volume as well. Do you know of any good competitive intel tools that provide accurate #s? I know Adgooroo is pretty good but it’s also pricey.

  17. Stan Chris says:

    Serpstat has launched a new one. In addition to knowing what questions your prospects are asking on the web, you can as well find out what drives traffic to your competitors and even analyze website structure.

Leave a Reply

Your email address will not be published. Required fields are marked *