Recent research was presented raising security and privacy concerns around URL shortening services like bit.ly, goo.gl, and others. The services are used to shorten lengthy URL’s to more compatible URLs suitable for online use. Smaller URLs also provide ancillary benefit since they are easier to remember. My first impression was the recent research on URL shorteners was that it was specious since URL shortening was never intended or designed as a security and privacy control from the start. Reading the research softened my initial opinion. The seeming randomness of these short URLs provides the public unfounded confidence of their utility for security. Specifically, the false idea that others will not discover the link since it appears secure – difficult to guess. Unfortunately, the part of the URI providing the identity for the long URL, is as few a 6-characters for some shortening services, far too small a space to be cryptographically secure, and easily brute forceable by attackers and demonstrated by researchers.
The research paper was not the first cracks in the short URL armor. The following presents some concerns I gathered across different resources from other researchers. I also share some personal thoughts about short URL weaknesses that I have not noticed elsewhere. I don’t stake any claim to these and I’m simply passing them along to raise awareness. I’m betting we have not seen the last around security and privacy concerns with short URLs.
1) Short URLs not secure
As researchers mention these links are not secure and easily brute forced. This may or may not be a concern for you depending on how you use them.
2) Short URLs target host unknown until clicked
Phishing is a problem for everyone. Short URLs exacerbate an already bad email phishing problem. There are some services like checkshorturl.com where email users can unwind these URLs but most people will never do this. People are trusting and verification takes extra work. Clicking a shortened URL is like hitchhiking in a strangers car, you don’t know where it’s taking you.
3) Obfuscated redirects
Brian Krebs makes an interesting point, attackers can leverage an open redirect on a government host and create a short branded URL. The result is an authentic URL that looks like it navigates to a government web site but instead navigates to the attackers malware site.
Becomes this branded URL (notice the .gov domain, ouch!)
The combination of an open redirect and short URL branding creates a situation of misplaced trust or false sense of security. Users think clicking will take them to a government site when if fact it takes them to another site entirely. The moral of the tale, if you have any open redirects in your web site your in trouble but if you also use branded URL shorteners your setting the public up for malware and phishing attacks.
4) Obfuscate payloads
A spin on Krebs idea I considered is that any arbitrary payload can be saved in a long URL by attackers and hidden from prying eyes – a payload. For example, on some services it’s possible to create arbitrary URLs with invalid hosts and parameters so long as those URLs are syntactically correct. Meaning if I create a URL https://www.xyz.com/def some shortening services are not checking to ensure host xyz is a valid host. Even if the host is valid, URI parameters may be developed that legitimate hosts ignore entirely like the following, https://www.xyz.com/def?a=b,b=c. Some servers like Blogger ignore superfluous parameters like a=b,b=c in the request if you pass them. Attackers can create any URL they want. I used the following in a quick test,
http://www.xx100kaass.com/0 (x10,000 zeros, for a 10k URL)
I created a bogus URL with a 10KB URI that consisted of a slash (/) followed by 10,000 zeros and was successful. Attackers can store payloads in these bogus URLs to use for a variety of purposes. Outside of validating the syntax and host, shortening services have no idea of knowing if these URIs are valid and, in their defense, there’s probably not a good way for them to validate. Therefore, they must store the entire long URL. This means an attacker can use URL shortening services, to hide small chunks of arbitrary data for nefarious purposes like command and control for bot networks, torrent information, etc. URL shortening sites undoubtably provide security intrusion and content controls. There’s likely some limits in size or number of URL per second they will accept, etc. I’m not sure what they are but it’s likely they vary between shortening services.
5) Multiple indirection
Some of the URL shorting services will not accept their own URLs for a long URL but at least a few of them will accept shorted URLs of other services. Therefore it’s possible to create multiple levels of indirection. short URLs referring to other short URLs. How many levels can be created? I’m not sure. It seems like browsers must have some practical level of redirect control but I have no idea. I’m not sure if this serves a practical purpose yet but at the very least it complicates organizational IT forensics.
6) Infinite loops
I was wondering if I could create two or more short URLs referring to each other. To get this to working requires an understanding of the shortening algorithm such that the attacker can determine the shortened URI before it’s created. Or perhaps a shortening services that allows changing a long URL after the short URL has been created. This will allow an attacker to create short URLs that either directly or indirectly refer to each other. I didn’t spend much time looking at this. I tried to find some code online to see if there were any standard algorithms. I was thinking everyone may be leveraging an open source project so I could determine the algorithm easily. Nothing was obvious, I was not successful. Perhaps someone else may want to take this up. I’m not sure if the browser is smart enough to detect these types of infinite redirects or not. If not, it seems plausible it could be used to hang or crash the browser. Even if possible, I’m not sure this has any practical value for attackers anyway.
7) XSS in URLs
8) Shortener Service Unavailability
If the shortening services goes away temporarily or permanently it impacts the services anywhere shortened links are embedded. What happens to Twitter if bit.ly goes away? Not good. DDOSing bit.ly is essentially the same as DDOSing Twitter since a better part of Twitters content would be unreachable for users if bit.ly cannot respond. Bit.do has a big list of shortening services. Bit.do also tracks shortening services no longer available and there’s many more of them I was aware. If shortening is part of your business strategy, or your users are using it, you may want to consider all your available options and weight risks, reliable services, hosting your own, etc.
Keep in mind my tests were not comprehensive and exhaustive. I didn’t want to do anything that could be considered offensive. So if noted a test was successful it may not be successful across all services. Conversely if a test was unsuccessful if may not be unsuccessful everywhere. An important consideration, while there are some problems with URL shorteners there’s not a good immediate option for avoiding them. If your going to participate in social media your going to be using short URLs like it or not until improvements are made.
* Landminds image from World Nomads
Updated on April 16, 2016
To understand why online systems are plagued with seemly endless security incidents requires a closer look into today’s security landscape. Let’s look first to understand the vulnerable systems criminals exploit. Top security company WhiteHat says it best on their home page.
According to WhiteHat web applications are the greatest risk area. Next WhiteHat says, “…most security budgets are spent on securing and monitoring the perimeter and endpoints”. According to the FBI 2014 Internet Crime Report, “…IC3 received 269,422 complaints with an adjusted dollar loss of $800,492,073…”, keep in mind this is US losses, not global.
“…IC3 received 269,422 complaints with an adjusted dollar loss of $800,492,0731…”, FBI 2014 Internet Crime Report
Aside from the claims and statistics, it does not take a security expert to understand the global force behind the online movement. Virtually every product and service is moving online and it stands to reason the criminals and crime are following the money.
Let’s change gears, let’s look into background on today’s top security executive the, Chief Information Security Officer (CISO). The following is Digital Guardian[INFOGRAPHIC] infographic for Fortunes 100’s top CISO’s.
The infographic tells us CISO’s are predominately male, well educated, hold various security and audit certifications. In short, nothing particularly remarkable outside of our expectations but take a look at the following, 59% of CISO’s have IT work background with only 13% in programming/engineering experience.
Fortune 100 CISOs are not well equipped with the skills necessary to defend today’s vulnerable web applications
Makes sense, for years IT leaders have been successfully defending permitters with firewalls. In all fairness, firewalls will always be valuable but they have not proven as effective defending online applications as well as IT infrastructure. Indications are Fortune 100 CISOs are not well equipped with the skills necessary to defend today’s vulnerable web applications. Let’s look at some of the reasons why.
Writing software code, software architecture, debugging, understanding the battery of tools, is an entire domain of expertise. Can programming be learned like any other challenge? Of course, but let’s give programmers some credit, application development is an entire domain of knowledge and takes takes years to master. Once that domain is mastered, learning to think like an attacker, breaking systems, secure coding techniques, secure coding libraries, dynamic and static analysis security tools are, in all fairness, is an entire new domain of expertise to master and not taught in most universities. A top defender of software and secure software designer is a unique skill set. This is why those that break into systems (e.g., pentesters) or secure traditional IT infrastructure don’t necessarily make the best application defenders.
Attacks occur where you least expect them and it’s often frustrating to newcomers in the application security profession
To give some idea of the learning challenges, learning basic programming principles like writing a “Hello World” program in Java will take about 10 minutes of time. Learning object oriented design techniques principles, some months. Learning the various Apache and open source packages you need to be competitive in a business environment can take years. Understanding how to defend all that technology takes years of working through incidents, developing the security mindset, understanding the tools and techniques. A strong technical leader requires mastery of two domains, software development and security. If you wanted a leader for security engineering this is all you would need but you don’t, you want a CISO. Now you need someone who also knows how to frame security challenges to smart executives and board members that may not be very technical. Strong CISO are rare individuals in high demand.
Today security is largely a software quality problem that can’t be addressed with the next vendor security-in-box-solution. Software security is a business and engineering quality problem – not an act of God. Software code must be designed, built, and delivered securely. Each step in the software development process, inception, architecture, development, testing, deployment, sunsetting, is important in the overall solution quality and historically entirely within the domain of software engineering groups. Let’s face it, software engineering leaders don’t necessarily appreciate security advice around how to build systems. Especially when the suggested security quality improvements reduce execution tempo which is closely related to performance based compensation.
Today security is largely a software quality problem that can’t be addressed with the next vendor security-in-box-solution. Software code must be designed, built, and delivered securely
Significantly reducing business risk depends on the CISO’s ability to influence and win the support of software developers, development leaders, business executives, and board members. Even a CISO with the best background and skills may not be able to influence positive code quality security improvements. A CISO is not an army of one. A knowledgeable CISO will fail without the proper support across business constituencies. This is because security is everyone’s job, not only the job of the CISO and their staff. Influencing systemic positive change throughout an organization is difficult but it begins with role dependent education. Today’s CISO’s must be as comfortable reviewing and recommending security architecture to a developer on the whiteboard as explaining business implications of security vulnerability to corporate boards. CISO’s must explain why engineering quality processes must be improved and recommend specific improvements when requested. CISO’s with best blend of technology and business experience have the best chance for improving software code quality and influencing the most positive changes to security and winning respect of developers.
As our most valuable assets are brought online as Internet web applications, criminals abscond with our data while companies are busy tweaking firewalls. Many companies are squandering security investments prodigiously in the wrong areas. Indications are Fortune Top-100 CISO’s don’t have the best blend of skills and experience to defend software systems – the primary weakness.
The trend is that all executives share security responsibility in a significant security incident so the value of a knowledgeable security executive should not be underestimated
The best CISO defenders of tomorrow will be those with experience coding/programming, designing, shipping software products and services. If a security leader with a development background is not available – build one. Find a top engineering leader and begin building the security mindset. Send them to security conferences where executives congregate like, Gartner IT Security Summit. Understanding business implications of security, executive concerns around security, and how to communicate with executives are essential. Send them to SANS Institute to learn how to break software applications. Theory is helpful but hands on skills are essential. Attend security conferences like Blackhat, DEFCON, and others. It can take years to find the best leader and build out a team. Begin now, by investing in your own organization and growing some organic talent. The trend is that all executives share security responsibility in a significant security incident so the value of a knowledgeable security executive should not be underestimated.
Laws and regulations have not kept pace with growth of Internet technologies. No clear expectations have been communicated to the software industry or users of these services by policy makers. Executives have responsibility for protecting customer data but enforcement remains selective. In the most egregious incidents, top C-level execs have been terminated for poor cybersecurity (e.g., Target).
variety of electronics, embedded devices, and sensors to the Internet. As practical example, some makers of city street lights have Internet enabled their bulbs. On the surface, Internet lightbulbs appear as useful as Internet connected refrigerators but a distinct advantage is that these bulbs will alert a central office when replacement is necessary. In a city with hundreds, or thousands of street lights, a proactive message of an inoperable light eliminates significant effort driving around to check bulbs. Even in the mundane case of the refrigerator, if Internet enabled, new water filters could be ordered before needed saving homeowners some trouble.
|Photo: Raspberry Pi RLT-SDR receiver|
What can you do with a RLT-SDR receiver, dump1090 software, and a Raspberry PI? Easy, you can capture data like flight numbers, altitude, speed, and position information from ADS-B equipped aircraft in your area.
Ever since I was a kid I always enjoyed listening to my Grandfather’s shortwave radio. Every twist of the tuner knob would produce a new discovery, aircraft, beeps and bleeps of Morse Code, far away news broadcasts. Now I’m much older and technology has matured but so have my skills. Now I have I have a Raspberry Pi, RLT-SDR, and know how to program (which means I’m dangerous).
Awhile back I built a Raspberry Pi project with a 2.8″ display from Adafruit. I also purchased a low cost RLT-SDR receiver at DEFCON 22. Shortly after I built my Pi project I could not make up my mind what I wanted to do with it so it sat on my shelf collecting dust. Same goes with the SDR receiver after I returned from DEFCON. That is until yesterday evening when I had the bright idea to put the Pi and SDR receiver together and make it do something useful. Around the time I was searching for more information on Internet to get SDR going on Raspberian I discovered some information about ADS-B. ADS-B equipped aircraft transmit telemetry on 1090mhz and within my SDR receivers bandwidth. You can learn more about ADS-B on RLT-SDR.com. I still learning myself so I don’t have a good idea where ADS-B fits into aircraft management just yet but ADS-B is definitely interesting technology.
As you can see in the picture of my Raspberry Pi screen photo (first photo), various information about aircraft flying in my area are presented near my home. I had no idea if this information was accurate so to verify I opened FlightRadar24 in my web browser (2nd photo on right). The results were accurate although I only receive a fraction of the plans arriving or departing from San Francisco, Oakland, San Jose, and Sacramento. I’m still not sure what the negative longitude is just yet. In any case, I noticed that I receive telemetry from some aircraft almost 100 miles away with a 4″ antenna – wow! Other aircraft passing over the mountains near my home would drop off my display and to be expected since mountains interfere with radio signals. I was very impressed with the unit I purchased at DEFCON from Hacker Warehouse and at $20US there’s no reason not to experiment. I noticed at the conference Hacker Warehouse sold a larger microwave antennas at the conference as well as directional antennas which would be interesting to experiment with.
The software package on the Raspberry Pi that makes detecting ADS-B transmissions possible is dump1090. Dump1090 is used to tune your RLT-SDR radio and receive the ADS-B. If you want to get dump1090 running on your Pi I then recommend reading Ferran Casanovas blog. Following are the command line options for dump1090 so you can get some idea for what it does.
My experience with dump1090 was excellent. The output is more or less what is shown on my Pi photo (first photo). I say, more or less, since I made some changes to the program for my smaller display on my Pi. The problem I had on my 2.8″ screen was that the lines would wrap around past the edge of the screen and into the next line. All the information was on the screen but it was hard to read in –interactive mode. To get the Pi display cleaned up I was thinking I could find a command line option and then grep something together for a cleaner display. Unfortunately, I didn’t notice any easy way to do this. As a workaround, I made some changes to the program to shorten the output to only the fields of interest within interactive.c. The code is customized for the 2.8″ PiTFT Mini Kit at Adafruit. After apply the changes, I recompiled dump1090 and output was shortened to fit my display as I expected. Next, I made some changes to force the Pi to login automatically and start the dump1090 program running. I know, not very secure but I don’t have any data on this device. For now, I just used the default account on the Pi but it would be more secure if I created a new account with less privilege. Anyway, I was lazy and wanted to get this thing finished before I went to bed so I improvised.
One final thought I have rolling around inside my head, since my profession is application security, is that ADS-B does not seem very secure. ADS-B telemetry is sent from aircraft real-time in route completely unencrypted so far as I know. I wonder what would happen if an ADS-B transmitter was built and launched in a ballon or drone by an adversary? It seems possible for adversaries to fake flight numbers, altitude, air speed, and position at a minimum. Transmitting on ADS-B band is more than likely highly illegal but then again adversaries give little regard to laws. I hope critical air traffic management systems don’t use these signals for routing traffic but I really have no idea. If anyone is an ADS-B expert and would like to post a comment to educate readers please do. I’m a noob in this area.
Update March 4, 2015, I have since learned other security researchers consider insecure ADS-B a security safety problem, Air Traffic Control Systems Vulnerabilities Could Make for Unfriendly Skies [Black Hat]. Apparently the Government Accountability Office (GAO) is recommending improvements, FAA Must Address Cyber-Security of Air Traffic Control Systems: GAO.
Update April 22, 2015, I discovered a presentation on the ADS-B at a security conference about 2 years ago, “DEFCON 20: Hacker + Airplanes = No Good Can Come Of This“. The presentation is provided by Brad Haines, Render Man(@iheckedwhat). Render Man goes a step further to demonstration ADS-B spoofing and does a simulated pass by an airport tower. The radio transmissions were terminated into a dummy load so no danger of harming any real aircraft. According to Render Man, FAA representatives where attending his conference session.
Update May 1, 2015, FAA’s answer to aging air traffic infrastructure is NextGen. Apparently, NextGen is falling short of expectations. A little digging on NextGen reveals it’s not the deep overhaul expected but more of tune-up. In fact, NextGen still includes proven insecure technologies like ADS-B. Unfortunately, the FAA efforts seem to focus on efficiency and safety as opposed to security which is a distinctly different challenge. FAA continues to press forward with NextGen even after debate on public research and the GAO report noting security concerns. The price tag for NextGen is around $40 billion but a complete overhaul would likely cost far more. New infrastructure or sharing military infrastructure may be required to develop a secure solution since foundational technologies like GPS were proven insecure long ago.