February 12, 2004
Deutche Boerse's "Competitive Intelligence" system
The previously reported Deutche Börse-originating spider has visited other webloggers as well, could potentially be a part of their competitive intelligence system "EXOTIC".
Markus Westner reports more information (in German) here.
Indeed this seems to be a robot/spider, mining news sources, weblogs and industry websites for information about companies & competition.
As seen in www logs, by pretending to be something it's not (various browsers / OSes) I'd dare to claim it's falsely pretending not to be a spider. Although it does read robots.txt, it is unknown which robot name it responds to (giving out normal browser names, it doesn't give any uniquely identifying trace away except IP address and odd behavious), and I didn't immediately find the creator/operator contact details in the web robots database.
We're on the case & will report back when we know more.
Referrers to this page
TrackBack URL for this entry:
: Competitive Intelligence (February 14, 2004 09:56 PM)
"The Deutsche BÃ¶rse has a competitive intelligence systems spidering various websites (found at Anders Jacobsen. Some information is available as a PDF presentation and on Markus WestnerÂ´s site (in German). Note: all following links in this post are G..."
24142 visits (6 today, 14 this week)