Toronto Internation Film Festival

Once again this year, Kristi and I are off to the Toronto International Film Festival (TIFF). We won't be hanging with any of the stars, but we will be watching a lot of films. Here's our schedule.

We've got tickets to 17 films right now and the 1st and 2nd choice calendars are things we'll be aiming to get tickets to. So far I'm most excited about:

• Accident (Yi Ngoi) - hopefully an awesome Hong Kong crime drama


• Whip It! - Drew Barrymore's directorial debut with Ellen Page staring as a misfit who turns to roller derby. This can't possibly be bad!


• A Serious Man - the Coen brothers go back to Minnesota in 1967 and hilarity ensues. We don't have tickets yet, but hopefully we'll snag some at the last minute.

Diagnosing the Diagnostics: Domain Summary

This is the second part of a series about Google's Safebrowsing Diagnostics page.

The first section of the diagnostics page (What is the current listing status for my site?) displays a summary of the status of the domain. It indicates whether the domain is currently listed in Google's Safebrowsing malware list. It may also list the number of times in the recent past that the domain has been added to and removed from the malware list.

Both these simple facts can be somewhat surprising...

A user or webmaster may see a malware warning in their browser when visiting a page. But the diagnostic page for that site may state that the domain is not currently on the malware list. This is usually a Cross-Site Warning. The domain is almost certainly infected with malware because users' browsers blocked requests to other malicious domains.

Webmasters may also find that their site has been added to the malware list numerous times. This may be an indication that the webmaster is cleaning up the infection on the server without closing all vulnerabilities. When Google's scanners process the malware review, the site is temporarily clean, so it is removed from the malware list. But because the server is still vulnerable, malicious content may be re-injected within hours. Google's malware scanners will quickly detect this and put the site back on the malware list.

Finally, for very new infections, the diagnostics page may be temporarily out of date. Google immediately flags sites found to be malicious and sends email to the webmasters. An alert webmaster who quickly notices that their site has been flagged may find that the diagnostics page lists their site as clean or even completely unscanned. The diagnostics page will update shortly - usually within a couple of hours.

The next post will deal with the most important section of Google's Safebrowsing Diagnostics page: What happened when Google visited this site?

Diagnosing the Diagnostics series:

• Background
• Domain Summary
• More to come...

Diagnosing the Diagnostics: Background

This is the first part of a series about Google's Safebrowsing Diagnostics page.

Google's Safebrowsing Diagnostics pages are a valuable source of information for webmasters. Unfortunately the information can sometimes be difficult to understand. Hopefully the next few posts will help clarify - I've been struggling with them for the past couple of months since they're fairly complex. Bear with me and jump in with any questions you may have.

Before we get to the diagnostics pages, we'll need to understand how malware works. Malware distributors have created sophisticated infrastructures. Typically, they use three types of servers and domains:

• Distribution Servers: Malware distributors usually have a set of servers that they control and use as a base of operations. The servers host malware tools and exploits and are usually on a network where they are unlikely to be shut down. They often don't have domain names associated with them, only IP addresses.
• Compromised Domain: Malware authors compromise legitimate domains and insert malicious scripts. The owner of the website usually doesn't realize this has happened. Most of the entries in Google's Safebrowsing malware list are these sorts of servers that have unintentionally become dangerous.
• Intermediary Domains: Between the distribution servers and the compromised domains there are often one or more intermediary domains. Malware authors establish domains to obfuscate their malicious code. For instance, they may use a common domain name with a slight misspelling, like gooqleanalytics (notice the Q), hoping that webmasters won't spot the typo.

The structure of Google's Safebrowsing Diagnostics pages mirrors these types of servers and domains. Most flagged domains will only be one type of server eg. compromised domains usually aren't also distribution servers.

With that background, you may already be able to understand the diagnostics pages a bit better. In future posts we'll dive into the details of each section of the diagnostics page.

Diagnosing the Diagnostics series:

• Background
• Domain Summary
• More to come...

Suspicious and Really Suspicious

Google's Safebrowsing Diagnostic page lists "the last time suspicious content was found on this site". But what does "suspicious" mean?

Google's automated malware scanners have been highly accurate with an astonishingly low false-positive rate. Part of that success has been because their definition of "suspicious" actually means "has nasty malware". If the scanners aren't really sure that a site has malware, they won't add it to the malware list. And that's the definition of "suspicious" ("has nasty malware") that Google's Safebrowsing Diagnostic page uses - content bad enough to get a site added to the malware list.

When the scanners do a review of a site to check if it should be removed from the malware list, they use a more stringent definition of "suspicious". If there's any suspicious activity at all then the site will not be removed from the malware list. Often sites have been infected with malware in multiple ways and the scanners need to be sure that it has been thoroughly cleaned up.

Those different definitions of "suspicious" may cause confusion when looking at Google's Safebrowsing Diagnostic page for a site that has been reviewed. The review may have found "suspicious" content that was not "suspicious" enough to have added the site to the malware list - but it is "suspicious" enough to prevent it being removed from the list. Google's Safebrowsing Diagnostic page won't list the date of that review scan.

If you're looking for the status of a malware review, log into Google's Webmaster Tools - the same place you reqested the malware review. It will show whether the review succeeded and will list urls that were still found to be "suspicious".

Cross-site Warnings

Several browsers use data from Google's malware list to protect users. Firefox 3, Chrome and Safari all check sites that users are visiting against Google's list and warn users if they are about to visit a dangerous site. There are some small differences in implementation across browsers that can cause confusion.

All three browsers check the address of the top-level page a user is navigating to. That protects most users in most cases. But, a web page can include content from another web page and if the included content is malicious then users may be exposed. Chrome (and Safari*) check every request against Google's malware list. This means those browsers will protect users even if malicious content from a flagged page is embedded on a non-flagged page.

Although that approach provides better protection for users, it may be confusing for webmasters if content on their site comes from another site. Some users (those with Chrome or Safari) will get warnings even though the webmaster's site is not blacklisted. Because the webmaster's site isn't blacklisted, they won't be able to request a malware review via Google's Webmaster Tools. Fortunately, this situation usually doesn't exist for very long. Google's scanners have already identified the embedded content as malicious but they haven't yet flagged the webmaster's site that includes the dangerous content. As they continue to crawl the internet, the scanners will quickly flag the webmaster's site.

If you're a webmaster in this situation, you'll need to examine all the content you're including from other websites. Look carefully at the warning page that browsers display since it usually includes the name of the domain that caused the problem.

* I can't say for certain exactly how Safari behaves because I haven't seen the code. But based on observation, Safari seems to have adopted the approach of checking every request.

Updated: FF3.5 checks every request against the blacklist and helps better protect users. FF3.5, Chrome and Safari all behave the same now.