So in this scenario, my competitors can spam links for my website and put me into trouble. Google will notice that my website has some spammy links (which my competitor created), and then penalize me for it. Then I have to contact every website that links to me to ask them to remove links to my website... else their website would get penalized too. I get such emails every week.
Rather than these fear mongering tactics, why can't Google just ignore spammy links instead of penalizing the whole websites and all the websites that link to them?
I didn't really like matts answers, some of them are not answers at all.
But! You may have hit the nail on the head. if i had to guess i would say that storing exceptions means that suddenly the algorithm had to check part of the corpus for these exceptions. at the scale at which google operates this could probably lead to rather big size and speed implications.
When monopolies distort markets, the impact of those distortions have outsized impacts on the rest of the ecosystem.
Ask Google engineers how they feel about centralization of power in telecom or finance or politics & they will tell you it is the worst thing ever
http://www.youtube.com/watch?v=Ik1AK56FtVc
Yet that same sort of centralization is fine when it is Google.
Those same engineers will admit that any algorithm has some level of false positives & false negatives. They may try to minimize these, but they can't make them zero.
The other thing which gets very little coverage, but is crucially important is that in spite of already having a monopoly market position Google keeps buying search marketshare with: their secret Android partner contracts, Firefox default search placement, iOS default search placement, Flash embedded into Chrome (so security updates happen in the background without sending you to a download site where trash can be bundled in with the update) & Flash security updates that hit all other web browsers coming bundled with Chrome bundleware which sets Chrome as your default browser, etc.
If Google decides they don't like you (for any reason), then you need searchers to use a different search engine in a browser which isn't paid off by Google for default placement. And you need to hope that users are savvy enough to repeatedly say "no thank you" to the automatically bundled Chrome install with their frequent Flash updates.
Google mentions how anything is only a click away or similar, but most people tend to use defaults.
When someone changes those defaults on Chrome users, that's a horrible user experience & the "hijacked" settings must be reset.
http://chrome.blogspot.com/2014/01/clean-up-your-hijacked-se...
But when those defaults are paid for by Google or changed through Google bundleware it is "a great user experience."
If Google believes their marketshare is a reflection of their superior search offering, they are welcome to stop buying default search placement in other web browsers & stop bundling Chrome installs on Flash security updates. But currently they are spending north of a billion Dollars a year on these activities, which indicates they clearly feel there is significant value to them.
Google's algorithm already analyze if a link is useful or spam. They can simply ignore the links that are spam and count only good links. But, Google decides to punish the websites caught in the fire. If your website has a link Google does not like, Google can penalize your website along with all the websites that link to you or you link to them.
> why can't Google just ignore spammy links instead of penalizing the whole websites and all the websites that link to them
You are suggesting an approach where there are no downsides to link spam. Obviously, players of this game would then ramp up their link spam bots, and run them non-stop. Because it causes no damage to the site they want to rank.
Link spam isn't just a search engine indexing problem, it is also a detriment to the general Web user too. It makes the Web worse, not just search.
Rather than these fear mongering tactics, why can't Google just ignore spammy links instead of penalizing the whole websites and all the websites that link to them?