This is kind of what the google and yahoo spiders of old use to be, programs that would scrape your site for key words and the like. Not we all like these because they give us legit hits to our web sites, but these spiders aren't all that friendly.
Semalt.com hits.
If first found out about these bot when this showed up in my Apache server log files.
201.50.251.78 - - [18/Sep/2014:10:47:06 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
89.154.210.157 - - [21/Sep/2014:05:16:27 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
201.34.81.160 - - [22/Sep/2014:05:37:09 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"
189.46.62.91 - - [22/Sep/2014:18:44:26 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
96.245.234.41 - - [23/Sep/2014:10:28:52 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
189.75.142.7 - - [23/Sep/2014:13:30:36 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
187.113.82.65 - - [28/Sep/2014:07:19:14 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
177.10.198.243 - - [28/Sep/2014:12:29:53 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
188.153.203.167 - - [29/Sep/2014:04:41:00 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
84.240.35.233 - - [29/Sep/2014:15:15:12 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
187.41.170.68 - - [30/Sep/2014:08:39:03 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
79.40.98.43 - - [30/Sep/2014:09:03:43 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"
186.233.117.92 - - [01/Oct/2014:09:28:48 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
201.95.200.9 - - [03/Oct/2014:12:16:13 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"
201.217.47.207 - - [03/Oct/2014:15:35:10 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
188.250.87.122 - - [05/Oct/2014:07:02:08 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
200.131.62.31 - - [07/Oct/2014:04:45:57 -0700] "GET / HTTP/1.0" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
179.252.38.142 - - [07/Oct/2014:06:28:36 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"
177.104.209.216 - - [10/Oct/2014:08:24:27 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
177.135.177.50 - - [10/Oct/2014:13:37:12 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
190.83.233.158 - - [10/Oct/2014:16:49:57 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
2.150.32.8 - - [11/Oct/2014:05:41:54 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
201.34.219.216 - - [13/Oct/2014:17:01:59 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36"
187.10.159.143 - - [14/Oct/2014:16:48:03 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"
186.215.131.42 - - [15/Oct/2014:10:00:38 -0700] "GET / HTTP/1.1" 200 11590 "http://semalt.semalt.com/crawler.php?u=http://kd7dmp.net" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
Now as you can they are all coming from different addresses, and when I did some digging on these address they where all residential fixed line or Moble ISP IP most of them coming out of Brazil. I thought this was very and seemed much like a bot attack. So I followed the link.
This sure looked a bit shady.....no info about the company and a page source code filled with data collecting java scripts.
So I did some more digging on Google and was surprised to find a very active account proclaiming to be the manager of the company. Some more queries and digging I found people all over reporting their stats being messed up by very large numbers of hits proclaiming to be references from this site. But those complaining had not paid for this service. Most of these where responded to by the manager sending them to links or vague treatments. See below with my Tiwtter feed about this.
The official Nataliya linked me to a page basically saying, "hey man this is the net, deal with it." But I would ask you to go look at it your self and make a judgement.
I went ahead a filled out the form on this page for one of my sites and I haven't seem the bots for a bit now, but that is only one day.
When Google says bad things about you.
With some more digging on Google I found several reports on Semalt being connected to a malware in a software called Soundfrost. Links to these article below, as they do a great job covering this.
Then more I dug into Google the worse it looked. I found out that the area of the world where Soundfrost was most used..... wait for it.... Brazil.
The hard facts.
So with all this in mind I went about proving this my self. Because.... why not!?
I used VMplayer and installed windows XP SP3 on it. On my main system I was running wireshark monitoring all the traffic from the VMPlayer's emulated NIC. I then went on to find a copy of Soundfrost to see what would happen.
Going to soundfrost.org I was able to download a copy, and nothing happened. It seem this copy didn't have the malware on it.
So I then went to http://soundfrost.en.softonic.com/ and found another copy of the software, this was infected and started talking to the mother ship right away.
After a bit of digging I found the proof I was looking for.
As you can see from the screenshot the same traffic I saw on my web server my honeypot VMplayer was sending to another server. So, how does it work? I only have part of it figured part of it out and hope to get some more info as time goes on.
How site hits work.
There are two programs that seem to do the most communicating, the Controlagent.exe and the ControlContent.exe. Both start out by connecting to soundfrost.com. The controlcontent.exe did a GET request for updates and keep getting a 0 back. The Controlagent.exe seem to do most of the work on this time around.
First it would set a GET request out to soundfrost.com/get_link.php, Then it got a error 302, and redirected to server19.soundfrost.com. Here it was sent a link like this.
http://semalt.semalt.com/semalt.php?u=http://ozzibylittlelotus.com
Then It would follow this link, and get something like this back.
<html>
<head>
<title>...</title>
<meta HTTP-EQUIV="Content-Type" content="text/html; charset=windows-1251">
<script language="JavaScript">
window.onload = function() {
var myEvt = document.createEvent('MouseEvents');
myEvt.initEvent('click', true, true);
document.getElementById('myLink').dispatchEvent(myEvt);
}
</script>
</html>
<body>
<a id="myLink" href="http://semalt.semalt.com/crawler.php?u=http://ozzibylittlelotus.com">Redirecting ...</a>
</body>
</html>
It would then follow this link and get this back.
..<html>
..<head>
..<title>...</title>
..<meta HTTP-EQUIV="Content-Type" content="text/html; charset=windows-1251">
..<script language="JavaScript">
..window.onload = function() {
..var myEvt = document.createEvent('MouseEvents');
..myEvt.initEvent('click', true, true);
..document.getElementById('myLink').dispatchEvent(myEvt);
..}
..</script>
..</html>
..<body>
..<a id="myLink" href="http://ozzibylittlelotus.com">Redirecting ...</a>
..</body>
..</html>
Then it would follow the final link and send the info as we saw up in the wireshark capture.
From playing around with this using curl I was able to figure out that the whole process allows the backend a very simple kind of handshaking with out any sensitive data being passed. So if the semalt.php doesn't get the same URL that get_link.php passed to the bot, it does nothing. And like wise the crawler.php does the same thing.
Now I wasn't able to get curl to work with crawler.php as it seem to be looking at the user agent and doing some kind of Java based redirect to the target site in question.
What's the point?
All in all, I'm not sure what the end goal of this is. It seems like they are making some money with this as I see reports of people using their service, though I don't know what happens after that. Other article I've seen indicate that the company uses artificial "clicks" to boost sites, but then why give that away for free?
On top of this they are using a botnet of a fair size, estimated to be in the 100,000 range.
So with all this why just fake clicks? I don't know yet.
The servers seem to be well locked down, nothing that makes it clear who may be running things behind it all. I have found some links to a Ukrainian company and the Russian search engine Yadex. But the connection to Yadex seems to stop in that they use them for analytic.Any one else that has info on this Please share and so that we can find out more about all of this.
Maybe they are don't have any other plans at all, but they have done a lot of work just to give away fake clicks.....
Very nice article, thanks for more information bootstrapreviewworld
ReplyDelete