Bing Predicts Is Unimpressive in NFL Week 3, Tries to Remain Relevant in Week 4 and Beyond

Note: This post is for the 2014 season. Click here to see the Week 3 results and Week 4 preview for 2015.

It’s long been said that any NFL team can win on any given Sunday. Microsoft’s Bing is out to prove that it knows exactly who is going to win each week. So far, it’s just doing an okay job.

After slipping a bit in Week 2, Bing Predicts and its virtual assistant Cortana climbed back to 10-6 for Week 3, matching Week 1's record. The prediction engine now has an overall record of 29-19 on the season.

Week 3 in Review

bing predicts week 3

We’re starting to see more patterns emerge from the search engine. In particular, the more confident Bing is with a prediction, the more successful the search engine is at picking the winner. Given the amount of disparity between the best teams and the worst teams in the NFL, this isn’t all that surprising.

During Week 3, Bing dominated in games where the predicted chance was above 70%. The search engine went a perfect 5 for 5 in these games, bringing it to 11-2 on the season. Before you go betting your life savings on every team with a 70% chance of winning, remember this is still a very small sample size. Yes, the search engine looks good in these instances, but there’s still plenty of room for error. Besides, you’d also have that pesky spread to consider. Bing just predicts winners, not which teams you should place your Vegas bets on.

Rapid Fire Stats

One of the oddest things about Bing’s Week 3 predictions was the number 58.2. The search engine listed 6 of 16 predictions with a 58.2% chance of winning. Bing was just 2-4 in these six games. What exactly does this mean? Well, besides telling you not to bet on teams Bing predicts with a 58.2% chance, not much. If we had more data, it could possibly lead us to crack Bing’s prediction algorithm, but we aren’t there yet.

Here are some quick stats from Bing’s first three weeks:

  • Bing is 22-13 when picking the home team to win
  • Bing is 7-6 when favoring the visiting team
  • Bing is 9-11 on the season when the favorite has a 50-59.9% chance to win
  • Bing is 6-3 in games involving the NFL’s three remaining undefeated teams (Bengals, Eagles, Cardinals)
  • Bing is 8-1 in games involving the NFL’s three winless teams (Buccaneers, Jaguars, Raiders)
  • Bing is 7-2 in games decided by more than 20 points
  • Bing has predicted three teams to win every week: Patriots, 49ers, and Seahawks. The search engine is 5-4 in these games.

Does the Final Score Matter?

One of the biggest surprises in the NFL this week was the final score of the Patriots-Raiders game. While it was no shock the Patriots defeated the visiting Raiders, most people probably expected a much different score. Tom Brady and the Patriots potent offense managed just 16 measly points against one of the worst teams in the league.

Going into this game, Bing listed the Patriots as having an 82% chance of winning. This marked the highest chance a team had to win yet—at least according to the search engine. Elsewhere on the web, one NFL prediction indicated a Patriots win over the Raiders was the most sure thing of the entire NFL season. What exactly does New England's measly 7-point margin of victory mean in light of these predictions? To put it bluntly, nothing.

The two biggest margins of victory this week came courtesy of the Falcons and the Colts. The Falcons rolled over the Buccaneers by 42 points, while the Colts defeated the Jaguars soundly by 27. What did Bing predict in these cases? 71.6% chance for a Falcons win and 64.4% for the Colts.

If the final score mattered, then Bing’s prediction in the Patriots-Raiders game wouldn’t seem all that impressive. In fact, one might jump to the conclusion that Bing "missed" this game, or that it somehow did a better job predicting the result of the Falcons-Bucs game.

Not so fast. We mustn't make the mistake of assigning a value that does not exist. In order to correlate a predicted margin of victory with the percentage chance a team has to win, we would need thousands and thousands of games in our data set. Even then, there might not be anything to it. When Bing says the Patriots have an 82% chance to win, they are essentially saying the Patriots would win 8 of 10 times under the given conditions. Whether they win by 7 points or 70 points, a Patriots victory means Bing got the result right.

Herein lies the problem with using Bing’s predictions as a way of betting against a spread. Bing’s prediction tool should not be used as a way to beat Vegas. An 80% chance of winning does not make a team any more likely to cover the spread.  At the end of the day, there is only one stat that really matters for Bing’s prediction method, and that’s the same stat that matters at the end of the NFL regular season (barring the need for any elaborate tie-breaking procedures).

Bing Fails to Impress but Remains Relevant

While 10-6 isn’t a bad record, it certainly isn’t an impressive feat. Other prediction methods, like Nate Silver’s Elo Ratings at FiveThirtyEight, performed better than Bing this week (12-4). Although Elo is more accurate, more transparent, and more in-depth, Bing might be a slightly more interesting study, if for no other reason then we aren’t really sure what it is doing.

One of the major mysteries is the social factor. Bing claims it uses the "knowledge" of the fans to help predict the outcomes each week. As we’ve seen in previous weeks, paying attention to the buzz around a team—like the Rams or the Ravens—doesn’t always translate into prediction success. As we look ahead to Week 4, there’s still no sign that using these signals is an indicator of a team’s likelihood of winning.

Week 4 Preview: Visitors, Ravens, and Change of Heart

Week 4 begins the NFL bye weeks, so we only have 13 games to look forward to this week. This obviously means fewer predictions, which means less data and less chance to declare something statistically significant. But that won’t stop us from having fun and over-analyzing everything.

Here is what Bing and its virtual assistant Cortana are predicting for this week:

bing week 4

One of the most interesting picks this week is the Ravens game. As of Monday night, the search engine had the visiting Panthers at a 64% chance to defeat the Ravens in Baltimore. Now, Bing is suddenly favoring the Ravens by 65.9%. Why the change? Both teams are 2-1, but the Panthers are coming off a big loss to the Steelers—the same team the Ravens easily defeated in Week 2.

The Ravens haven’t lost a game since the Ray Rice videotape was released, but this is the first time Bing has them as the winner. In the past weeks, the signals from the fans may have led Bing to list the Ravens as the loser. Last week, other prediction methods had the Ravens as winners with a 60% chance. It seems odd that Bing would have strayed so far from the pack unless it considers those social signals and fan knowledge to be a huge factor.

In other news, Bing is pulling for the visitor more than ever this week. In 5 of 13 games (39%), Bing has the home team losing. In four of these games, the visiting team has the better record. In 8 of 9 games this week, Bing is predicting the team with the better record to win.

Bing has changed five of its picks since Monday afternoon: the Panthers to the Ravens, the Jets to the Lions, the Vikings to the Falcons, the Chiefs to the Patriots, and the Saints to the Cowboys. In three of these five instances, Bing shifted from a Week 3 loser to a Week 3 winner. The Saints-Cowboys game is virtually a toss-up, with both teams coming off a less-than-impressive victory. Might as well go with the home team, right? Bing has predicted the Patriots to win every week, so this change is no surprise.

Bing’s surprise pick of the week: the 49ers over the Eagles. I don’t care if this game is in San Francisco. The Eagles (3-0) are soaring right now while the 49ers (1-2) are in a heap of trouble. Then again, this doesn’t look like a season where anyone will stay undefeated for long. The most interesting part of this prediction: Bing favors the 49ers with a 58.2% chance. We all know what that means…

This week’s locks: Bing has the Steelers beating the Buccaneers with a 78.9% chance and the Chargers over the Jaguars with an 83% chance. After this week, Bing should be 13-2 when predicting a chance of 70% or more.

I have the search engine going 9-4 this week. I’ve overestimated its abilities every week so far, but I’m feeling pretty good about this one.  Come back next week for a Week 4 summary and a Week 5 preview.

Week 4 Update

On Wednesday morning, Bing updated its picks for Week 4. There were two notable changes. The search engine now has the Raiders defeating the Dolphins, and it has switched back to the Saints over the Cowboys. What prompted these changes? It's hard to say. There hasn't been any major news coming from any of these teams. Perhaps Raiders and Saints fans have been stepping up their social game.

Here are the updated picks and percentages for Week 4, as well as a preview of the Packers-Vikings Week 5 game:

week 4 updated

First Scribe is now Perrill Why the change?