This is the first post in a series looking at competitive research tools and their data accuracy. Well be looking at Spyfu.com today.
Competitive research tools are meant to serve one purpose, to get accurate data on the competition and turn that into actionable items. So how do you know ifthe data that you are getting is actually accurate or not? Perhaps youve plugged in one or two of your own accounts, maybe youve just trusted the data, or maybe youve not cared. Well now is the time to care! Sharing is caring, so we are going to share data on several of clients and compare that to the competitive research tools data.
As of now I have NO IDEA what we will come up with.
First, lets get a little information about Spyfu.com and their features. If you want a full review of their tool Google spyfu review, however for this post we are going to focus on accuracy of data.
- Classic- allows you to see competitors budget and keywords.
- Kombat- gives you the keyword overlap of up to 3 competitors.
- Keyword SmartSearch- allows you to generate keyword lists and easily filter.
- Ad History- shows a month over month history of ads.
Down and Dirty with Data
Were going to take different accounts and test the following areas for accuracy:
Spyfu gives a range for the budget. We took the mean of the range to make the data easier to understand. Our numbers were taken from separate dates within the last two weeks.
There is obviously a significant difference in the budgets. This got me thinking that perhaps Spyfu is only tracking budgets on the search network. Here is how budgets look when only looking at numbers within search.
Average percentage difference was still 39% higher, but more accurate. Lastly, I took the average previous months budget and compared Spyfus previous months budget (found by hovering over the previous month in the chart).
This was the closest to being accurate. With seasonal and accounts less than 3 months old being most inaccurate.
- The previous months data, found by hovering over the previous month in the graph, was more accurate than the current months data, which is prominently displayed.
- Numbers were more accurate for accounts with budgets that were consistent and unchanging for several months.
- Seasonal clients and accounts with less than 3 months data were significantly off.
For keywords we took the entire list of active keywords in our accounts (no paused, low search volume, etc.), took out any match types (because it appears that Spyfu does not account for match types) and then compared our total number to their total number.
- It appears that the larger the account the larger the discrepancy. However, in those accounts there are also more long-tail keywords, which may account for that difference. I compared keywords for one account that had received impressions to Spyfus keywords and found that there were few in common. One possible conclusion for this is that broad or phrase match keywords were triggering other variations of keywords, relevant of course.
- Keyword count and accuracy are more inaccurate for larger accounts.
- Keyword lists are not 100% accurate so they should be not be assumed to be winners immediately(duh?).
While Spyfus data is not completely accurate there are still many benefits to using the tool for competitive research. Budgets are slightly off but can still be used to get an idea of where budgets in your industry are at. By comparing several competitors you can tell who is spending the most. The keyword tool can give you keywords to expand your account or get it started. The Kombat tool seems to be the most useful tool that Spyfu has to offer.
I’m interested to hear how accurate Spyfu.com is with your own data. What do you find when you plug in your own account(s)?
****Update – March 19 – Response From Spyfu****
Mike Roberts with SpyFu. This is cool stuff.
You’re definitely right that our budgets are search only; no content network, no partner sites.
When we publish the budget numbers we post a range. Something like $165-$330. The fewer data points we have the wider that range is; it’s an expression of uncertainty --> very much like presidential polling sampling errors (+-6%). The larger the sample size (inclusive of history), the more accurate the results.
I’m curious what you did to turn our budget into one number. Did you take the bottom number, the top number, the mean? [I took the mean -Luke]
I’d be curious how often your client’s budget falls within the actual range. [I added this to the chart]
In any case, calculating someone’s ad budget from what we see them advertise on is akin to predicting intelligent life on other planets based on the wavelength of their star. There are so many factors that determine budget (Quality score, bid strategy, position, cost per click [which we get from Google, but they don’t report accurately], Shopping, Videos, Images)
Here’s the thing: When we say someone advertises on a keyword, it’s 100% true — there’s a cache page you can use as an audit trail going back to 2006. Ad budget is the biggest extrapolation we make; the farthest from the raw data. My goal when I first calculated that number was to make it so that given that I know my own Adwords budget, can I estimate how much bigger (or smaller) a player my competitors are in the market. Then, secondarily, I wanted to be able to roughly gauge the size of a client, competitor, or partner; are they spending hundreds, thousands, tens of thousands, millions? It’s kind of hard to tell by looking at someone’s website, right? Make sense?
Anyway, I love what you’re doing. Thx.
photo courtesy of laweekly.com