Brief History

Throughout the course of three years I have been studying the Voice Over IP attack canvas. As a security engineer, I have had the privilege of seeing an enormous amount of attacks while working at a Managed Services Provider. Because I have multiple roles in the company I am working at, I get the best of multiple worlds. Sometimes I’m a VoIP engineer, sometimes a pentester, sometimes systems engineer, other times it depends highly on my mood. Overall I’m a Senior Coffee Tasting Engineer working at a company that gives me great flexibility to do what I want. This allowed me to create my own Voice Over IP Honeypot I called Arkeos for data analysis.

My original security application began as a “blacklisting” slash “VoIP Intrusion Prevention System” program using Asterisk. The goal was simple, keep an eye on live servers and when an attacker made a move, initiate a mechanism to automatically block the attacker, send the engineers a report, place the offender in a list to be distributed to all other PBX’s in my company so they too can block them. Initially my VoIP IPS worked just as programmed however, because I have clients all over the world, it became difficult to outright block entire CIDR ranges. I quickly whitelisted many addresses, re-hashed the IPS which worked fine for years.

The Arkeos idea came to mind after browsing around to see that there really wasn’t anyone to correspond with in this matter. People looking for and sharing similar data were few as opposed to normal networking slash abuse. I decided to build my honeypot based on what I was seeing from monitoring logs. Anyone who has known me for years can tell you that I kept tons of terminals opened with good old faithful tail -f running alongside whatever logging applications I have in house (Splunk, etc). As months passed by, I began archiving data from attacks spread across 20-30 Internet facing PBX’s. My current company sells VoIP trunks, managed VoIP services, T1-OCx connections, managed security services, video conferencing, digital signage, etc.; if it’s out there, then we likely do it. This gives me a high degree of visibility for not only VoIP attacks, but other attacks as well.

Arkeos is a program primarily used on Unix based servers running the Asterisk Open Source PBX. The application mimics a valid extension in which an attacker, after bruteforcing an account, attempts to place calls. What the attacker doesn’t know is that their calls go nowhere. The sole purpose of allowing them to perceive they have an account is to track them. Where they come from, what accounts they use, what VoIP account scanners they use, what numbers they try to dial. Because of the flexibility of Asterisk and Unix as a whole, the application does a lot more than stated however, I will not disclose too much otherwise any attackers reading this might catch on. Then I won’t have anyone to play with.

As time progressed customers who purchased SIP trunks from us were getting “owned” from time to time and to date, I have been called to handle incident response and forensics on over 75 VoIP servers ranging from Asterisk, Avaya, pbxnsip, CME and a mixture of everything in between. Usually the outcome has been the same: weak accounts and weak passwords.

2007 – 2008’s Attackers trends

For the better part of 2007 and 2008, attackers had a common modus operandi: scan a host for an open account, register a device (soft phone, another SIP trunk) and fire away thousands of calls. I have personally seen a company spend over a quarter of a million US dollars because of these attacks. Before I dive more into stats a bit, I will explain a bit about the financial loss for those who don’t think that it’s a big deal. I currently have a client who purchases a heavy amount of SIP trunks from my company, they in turn have a “Vonage” like business. Their customers buy an ATA, number, connect and place calls from Latin America. My client one day sends me an e-mail asking me if I would be willing to take a look at his network as his upstream was sending them abuse messages. Game on.

My client has approximately 20 servers in the continental US and 20 in Latin America. They were using a mixture of Asterisk and other equipment. What was happening to their servers was two-fold, attackers were leveraging VoIP accounts and placing thousands of calls. During the times when attacks took place, their bandwidth was so saturated that normal clients’ calls experienced horrible quality and drops. This led to loss of clients, enormous phone bills to places like Sierra Leone, Romania and the list goes on. To make a long story short, it took me approximately six weeks to clean up their shop as I could NOT take any of their servers offline. I essentially had to analyze what occurred, cut the bleeding, implement a fix and implement post remediation. Ultimately I programmed a distributed “cleaning” application to clean up accounts, create appropriate firewall rules, clean up backdoored binaries, implement an SIEM. This was another cost the client had to contend with. When I say over a quarter of a million dollars I am being very conservative. That was only the first instance of what occurred in their network, the second was, a few of their servers had become bots – but that is another story. Moving back into the statistics and trends of attackers I’ve seen, attackers are becoming smarter and the attacks seem more focused and targeted.

2009 – 2010’s Attackers trends

On a daily basis I see approximately six or seven attacks on each one of my Internet facing PBX’s, this estimate is based on solid numbers. The total amount of attacks divided by the amount of servers I have. While it may not seem like a big deal but the fact of the matter is, if one attack cost a company a quarter of a million in damages, the last thing I need to see is an attack on my company. Management’s first response would be “why didn’t you stop it.” The reality is, my company too was the victim of tollfraud via legacy VoIP accounts. We experienced 96 billable hours from our carrier to calls made to numbers in Sierra Leone. I know firsthand that these attackers are not “script kiddies” but focused fraudsters. Altogether during a highly volatile week, I can see approximately 150 – 200 attacks per day.

Previously (between 2007 – 2008) when I monitored the attacks, the attack phase was: “get an account, fire and forget,” whereas now, there seems to be less automation and more intervention. This is evident in the way attackers are placing calls. Here is an entry from today in my Arkeos database of an attacker coming from an address in Germany. Take note that I have sanitized data to keep the identity of my honeypot at bay:

# more peer-verbose-Mon-21-Sep-2009-11\:59\:34-AM-EDT.txt
 -- General --
           Name: SIP/300-b6b241c0
           Type: SIP
       UniqueID: 1253548774.7688
      Caller ID: 1xxxxxxxxxx
 Caller ID Name: xxxxxxxxxx
    DNID Digits: 16085541772
          State: Ring (4)
          Rings: 0
  NativeFormats: 0x2 (gsm)
    WriteFormat: 0x40 (slin)
     ReadFormat: 0x40 (slin)
 WriteTranscode: Yes
  ReadTranscode: Yes
1st File Descriptor: 39
      Frames in: 0
     Frames out: 0
 Time to Hangup: 0
   Elapsed Time: 0h0m0s
  Direct Bridge:
Indirect Bridge:
 --   PBX   --
        Context: xxxxxxxx
      Extension: xxxxxxxx
       Priority: 2
     Call Group: 0
   Pickup Group: 0
    Application: Dial
           Data: SIP/1xxxxxxxxxx
    Blocking in: ast_waitfor_nandfds
      Variables:

SYSTEMSTATUS=SUCCESS
SIPCALLID=146ef044580a3a795de94b541db21249@xxx.xxx.xxx.xxx
SIPUSERAGENT=Asterisk PBX
SIPDOMAIN=xxx.xxx.xxx.xxx
SIPURI=sip:300@85.214.85.180

  CDR Variables:
level 1: clid=xxxxxxxx
level 1: src=xxxxxxxx
level 1: dst=xxxxxxxx
level 1: dcontext=xxxxxxxx
level 1: channel=SIP/300-b6b241c0
level 1: dstchannel=SIP/xxxxxxxx
level 1: lastapp=Dial
level 1: lastdata=SIP/xxxxxxxx
level 1: start=2009-09-21 11:59:34
level 1: answer=2009-09-21 11:59:34
level 1: end=2009-09-21 11:59:34
level 1: duration=0
level 1: billsec=0
level 1: disposition=NO ANSWER
level 1: amaflags=DOCUMENTATION
level 1: uniqueid=1253548774.7688

So what occurred here? The who, what, where and how? The four most important variables are easy to parse:

How: Name: SIP/300-b6b241c0
Where: DNID Digits: 002314599711
What: SIPUSERAGENT=Asterisk PBX
Who: SIPURI=sip:300@85.214.85.180

How? An attacker leveraged a SIP account on the honeypot (username=300) Where? They tried dialing 16085541772 What? They likely tried trunking the SIP account to their Asterisk box Who? 85.214.85.180 So who owns the server? A company reselling hosting:

inetnum:        85.214.16.0 - 85.214.139.255
netname:        STRATO-RZG-DED2
descr:          Strato Rechenzentrum, Berlin
country:        DE
admin-c:        SRDS-RIPE
tech-c:         SRDS-RIPE
remarks:        ************************************************************
remarks:        * Please send abuse complaints to abuse-server@strato.de   *
remarks:        * or fax +49-30-88615-755 ONLY.                            *
remarks:        * Abuse reports to other e-mail addresses will be ignored. *
remarks:        ************************************************************
status:         ASSIGNED PA
mnt-by:         STRATO-RZG-MNT
source:         RIPE # Filtered

Wondering if this is spoofed (since some seem to think it is):

# nmap -sU -p 5060 85.214.85.180

Interesting ports on server1.stacheshof.de (85.214.85.180):
PORT     STATE         SERVICE
5060/udp open|filtered sip

Nmap finished: 1 IP address (1 host up) scanned in 1.755 seconds

Likely not spoofed however, this does not stop someone from compromising a host and using it to do other things. The likelihood of this server being compromised AND running a PBX on it is low and here is my reasoning for this comment, I’ve seen this address quite a few times connecting to different addresses during different months. I would think a server’s administrator would notice a PBX server running on their system. What makes things even more interesting is the following, the attacker tries to make one call, then starts changing his (or her) dialing patterns:

002314599711 - 20100921-11:41:33
92314599711 - 20100921-11:47:42
9002314599711 - 20100921-11:54:31
90112314599711 - 20100921-12:00:29

The calls were placed at odd times to make me think they were automated. The first call was at 11:41 followed by six minute gap, followed by a 7 minute gap, followed by another 6 minute gap. In previous times, automation was instant. Even odder is, how is this person aware that a call didn’t go through and that they should be changing the dial plan? My response is, they have some form of visibility into that destination number. Calls from my honeypot sound just as any other call with the exception they all end up in one place. Voicemail to a non-existent account. This further allows me to hear the audio if any is left.

After accumulating all of this data, I figured it would be best to share it with others in the industry via the “VoIP Abuse Project.” The purpose of the project is to create a blacklist of sorts for any engineer or admin to be able to use to block out the bad guys. Alongside blocking out the bad guys, it always offers a place to showcase the realities of “abuse desks” of companies whose networks are being used in attacks. That goal too is simple, systems are attacked and reported, companies responding and correcting issues should be applauded, companies ignoring the issue should know that it will only be a matter of time before their clients cost them more in lost revenue. It’s their networks, they should be more responsible about what leaves their network. Anyhow, this has become rather long so I will leave this as a part one. Stay tuned for more.