Last Friday, Google mistakenly removed from its index the search engine of its index and search results. This meant that none of our pages appeared in the Google search – none.
Fortunately, we were able to get a response from Google on Friday morning. We were told that his "system incorrectly identified the site as hacking" and was therefore mistakenly removed from the index. Google said it solved the problem and solved the problem early in the morning, thereby removing the hack classification. However, our site did not return to the index around 17:30. that night.
The irony of a website dedicated to marketing coverage by search engines that has accidentally removed from the Google list has not been lost to us ( or the community). Below you will find more details about what has happened, how Google has answered our follow-up questions and, more importantly, about SEO, which webmasters can learn from our drama day.
What we have learned so far
In total, Google did not include the search engine in the results for more than 12 hours on November 30.
We are told that the site was completely deleted (instead of being notified) because Google's hacker classifiers had some sort of bug and classified our site as hacked then that he was not. "It was an error in our systems – a false positive stating that the site had been hacked," said a Google spokesman.
Why no warning before deletion? One of the main questions we asked from the SEO community was: Why did not Google just tag the site as being hacked into search results? Why did Google make the heavy decision to completely remove the site from the index?
"If we determine that a site is hacked, there are rare cases in which we can deindex the site," Google told us. "For example, if spammers degrade organic pages, including the home page, and completely replace the original content with spam. These types of hacking are rare. "
It makes sense if a site is so badly hacked that Google does not want users to run the risk of being infected with their computer after visiting a compromised site. So, Google completely removes these types of hacked sites from the index. But obviously, when Google makes a mistake, it can have significant consequences for the site concerned.
Do False Positives Happen Often? We asked Google if it was a frequent occurrence. That's going to de-index the sites because of bugs detected in hacking classifiers. "Wrongly reporting a site is something that should never happen," Google replied. "Unfortunately, we do not always do things right," added Google. Then Google explained a little more about how it works: "There are complex systems for tagging sites, as you can imagine. Automatically determining the hacked status of a website is not an easy task. "
"In this case, we were not able to detect the error in the backend before the site was removed from the list as being hacked from our search results," Google told us.
We know this is relatively rare, but Google has not indicated how often these errors occur.
Why did an e-mail take several hours? The Google Search Console is programmed to send automatic email notifications about site hijacking and other website issues. For some reason, in this case, it took several hours for this e-mail to be sent.
The site was probably deindexed by Google around 3 am Friday morning. We learned the problem of readers on social media. We have never received any warning or hacking notice that we needed to repair within a certain time before the site was deleted.
"We strive to be as fast as possible to ensure that website owners are aware of the changing research situation," said Google. "Ideally, the machines should work in unison and in real time. As you have seen in this case, the reality is a little different: there is definitely room for improvement here. "
Of course, we would like to have more answers. We still do not know what exactly caused Search Engine Land to be classified as a hacked site. Was it really just a random error or was there something that triggered the false positive? Google had not yet provided more details on this issue at the time of publishing this story.
We would also like clarification on the average time the Google Search Console needs to send e-mail notifications, especially notifications of serious problems such as the one supposedly detected on the search engine.
What happened to our traffic that day?
Of course, every site is different, but we're sure you can imagine losing organic search traffic for a tired day. But the good news is that other sources have gained a lot of attention as users and community members look for other ways to access our content. For example, compared to the average Friday, organic Bing traffic has increased by more than 15%. What really minimized the losses was the direct hits, the stops and the social successes. Direct traffic was up more than 25% above average. The number of referrals has increased by 90% and the traffic on social networks has exploded, exceeding 300%, mainly due to the daily coverage of our unfortunate process of de-indexing.
Honestly, most of these gains are directly related to our community, which has helped to increase our traffic from other sources.
What can you do if this happens on your site?
We'll be honest, it was not fun, but in the end, there will be no lasting damage. But one can not help wondering how tragic it would have been to experience a similar event on an ecommerce site, especially during Cyber Week. As digital marketers, we all know the precarious nature of using Google – or a particular company – to drive a significant amount of traffic on our websites. But examples like this highlight the inherent vulnerability of being subject to the vagaries or errors of a major source of traffic. Especially when there are no humans, you can call to solve a problem.
If this kind of error can happen to anyone, here is some advice that we can convey from this experience:
If you think you have been wrongly penalized, make noise in many places and hope to be heard quickly. Google has suggested Google Webmaster Support Forums where, it is said, there are hundreds of product experts and Google specialists who are reachable every day. "These people can make the problems worse on their behalf." But as someone who knows these forums quite well, it's often difficult to get things done through these forums. Use Twitter to be heard as well. This is Google's main account and, as a referrer, you probably already follow the accounts of John Mueller (@JohnMu) and Danny Sullivan (@SearchLiason).
If your organization is distributed, try to give people in multiple time zones access to Google Search Console.
Have a plan to take advantage of other channels to try to compensate for traffic losses. Tweets, for example, appear in Google search results. We did not have a plan and, again, we were fortunate that the news of our difficult situation caught the attention of the community. This ended up generating significant direct, reference and social traffic. Develop an emergency plan now – which should apply to any channel that generates a significant portion of your traffic or revenue. This will require coordination with content, social networks, email, advertising and other teams beyond digital. There are probably promotions, content items or ways to solicit influencers that you can consider deploying in case of an emergency.
This episode reinforces the following saying: "You do not want to keep all your eggs in one basket." Fortunately, with our active community, our presence on social networks and our general reputation, we came out relatively unscathed. But Google is still a black box, by its own admission, and its machines do not always work as promised. Try to diversify as much as possible and have contingency plans.
If we learn more, you will be the first to know.
About the author
Barry Schwartz is the editor-in-chief of Search Engine Land and is the owner of RustyBrick, a New York-based web consulting firm. He also directs Search Engine Roundtable, a popular research blog on topics related to SEM.