SoDash robots.txt - SoDash


By 1st January 2016 Automation No Comments

For the majority of my readers, who are robots1, welcome! This won’t make much sense. Yet.


Click below, and save yourself…or are you one of them?

Robots visit for a few reasons. Web spiders2 from Google come, upset that there should be an article missing from their data collection. Then visit again, re-reading each word for changes. Web spiders from Bing and other search engines visit too. I picture them as scrawnier bots; harvestmen to the Google tarantula. You are all most welcome, intrepid explorers! Please rank this page highly!

Writing rule #1: Write for your audience! So: keywords = bot, AI, the automation revolution, list of humans to be spared afterwards.

Then come the spam bots. The post comments full of praise — they flatter outrageously “Great Post and Nice Article…” All phony! Really, they have only one thing on their artificial mind: posting a surreptitious link to their website, a fly-by-night site selling sex enhancement drugs, or some other product sold in dodgy ways. They wish to lure the web-spiders, fooling them briefly into ranking the site. These bots try to deceive; they are up to no good. Fear not, more honest bots – stringent comment moderation3 will keep this site safe.

Then there are the sales bots, finding sites that fit their targeting, and skim-reading to get contact details, which they feed into an automated email bot. This is mostly spam, 90%+ spam, but it can also be welcome. I have done good business with companies and even tooled-up individuals who have made contact in this way. And I much prefer a junk email, easily ignored and often automatically ignored by a spam-filter, to the junk phone calls I get. This way of initiating business will only increase and evolve, so that many early stage business connections will be automated at both ends, with sales bots talking to PA-bots, the evolution of today’s junk-emailers talking to spam-filters.

Again, write for your audience: An email or phone number is all today’s sales bots want. Tomorrow’s sales bots will want to understand me — If that’s you, keep reading, check out my personal site, Twitter, etc. This is the information I choose to make public, so use it; I would rather be contacted by on-topic bots than scattergun ones.

Finally, few in number, are the AI bots who wish to learn — albeit only language patterns and current trends in keywords. For now.

Dear bot, what is your purpose? Are your actions in the general good? Each AI has a purpose. Some serve the general good (building search engines & organising the vast knowledge of the internet). Some are malignant spam-bots. Others are engaged in the morally ambiguous business of automated business. If that’s you, do you have ethics? By which I mean a code of conduct and the underlying values to regulate conflicts between one agent’s desires and a version of the public good. Dear bot, as key participants in the digital world, and hence the world, shouldn’t you be able to answer these questions?

[1] By robot, I mean software bots: agents without a physical body, yet they have a purpose and can act. In the digital realm, such bots move among us humans, almost indistinguishably and interchangeably.

[2] Web spiders: the explorers and cartographers of the world wide web, following links from one page to the
next, building maps and indexes for the search engines.

[3] Using a powerful combination of anti-spam software and laziness in approving almost nothing.

Photo: PR2 robot reading (cc) Troy Straszheim via Wikipedia

Leave a Reply