Product

8 Things To Think About Before Building Your Own URL Shortener

Tired of those long URLs taunting you on Twitter and Facebook?

Want to find a solution that brands and packages up your links in a way that’s engaging and click-worthy? Thinking about building your own link shortener?

With the average person’s attention span now at eight seconds – that’s shorter than how long a goldfish can hold a thought  – you need all the edge you can get.

But before you make the leap and build your own shortener, there are a few things to keep in mind:

1) Metrics

The digital ecosystem is becoming increasingly maze-like with multiple devices, channels, and walled gardens. Brands are struggling to collect, organize, and optimize the customer experience, with 87% of customers reporting that brands need to put more effort into providing a consistent experience.

The good news is that the link is an invisible thread which helps you see clearly across the internet. That is, when it’s powered with the right tracking and analytics tools on the backend.

But building an accurate, reliable analytics system on top of your link shortener is hard and expensive. In particular, it turns out that counting things at a certain scale is pretty hard.  Some examples of challenges that can come up sooner than you might expect include:

  • Bursty traffic 

    Really popular links tend to see very large spikes of traffic that quickly drops off. Unfortunately, you generally need to have enough infrastructure in place to deal with those spikes, even if it ends up sitting around idle much of the time.

  • Clicks add up

    Over time, the storage and processing power required to store and count all of your clicks adds up. At a minimum this growth will increase infrastructure costs over time and at worse, it may force periodic system rebuilds as solutions that work with smaller amounts of data often do not work well as data grows.

  • The internet is weird and always changing

    Do you know how to detect bot traffic?  How about identifying mobile vs desktop traffic? Even if you do know those things today, you will need to keep up-to-date with new developments and ecosystem changes to regularly update your system.

At Bitly, we’ve spent years investing in developing sophisticated analytics systems that provide reliable, consistent, and cost-effective metrics and continue to develop this technology every day. We also gain a number of economies of scale by tracking metrics for all of our customers. This means that we can invest in more capacity than any one customer could justify or saturate on their own.

2) Resiliency

Given the nature of shortlinks, it’s critical that any short link service be as highly available as possible.  

Unfortunately, there’s a lot that can happen to knock a service offline: power outages, network issues, hardware failures, denial of service attacks. These issues, and more, can disrupt service for your users and cost you precious resources to deal with.

We take reliability seriously at Bitly. Our team has spent the past six years making big investments to provide rock solid reliability, no matter what happens. Some of the measures that we’ve taken include:

  • World Class Service Providers

    Our services are primarily operated out of IBM Softlayer and are protected by Akamai Prolexic, both top-tier providers who have the knowledge and infrastructure to support highly available services.

  • Built-in Redundancy

    Within our primary data center, all services from the top to the bottom of our stack are designed with resilience and fault-tolerance in mind.  Additionally, we’ve built completely redundant but vendor, geographic, and implementation diverse redirect infrastructure that can take over if our primary data center becomes unavailable for whatever reason.

  • 24/7 On-call Team

    We have a dedicated team of highly experienced engineers who continually monitor our infrastructure. They are on-call around the clock and ready to take action to respond to changing conditions and keep our services running smoothly.

  • Global Monitoring

    We monitor the availability and performance of Bitly links from 80+ nodes operating on customer and business ISPs from around the world. This allows us to identify and address network issues ranging from isolated routing issues to major network events quickly and effectively.

3) Security

According to research by IBM and the Ponemon Institute, the average total cost of a single data breach is $3.79 million.

Being unprepared for a security breach can be costly. Fending off an attack means reallocating resources and possibly paying a premium to hire a third-party security vendor. It gets even pricer in the long run, with your reputation and customer trust on the line. As it stands, 69% of adults say they are not confident that records of their activity maintained by the social media sites they use will remain private and secure.

At Bitly, we keep our links secure by following industry best practices including:
 

– Storing passwords using strong, modern hashing functions
– Using XSS and CSRF protection tools
– Performing thorough review on all code that goes into production
– Logging and providing visibility into all account activity
– Restricting access to production systems to only those who need said access to do their work
– Making sure all code deploys through a production system that is actively logged and audited
– Using multi-factor authentication for internal services/tools as well as offering MFA to customers within our product
– Actively participating in Facebook’s Threat Exchange
– Developing and maintaining relationships with the security and abuse teams at major social networks and service providers

We take our position as a trusted service provider and a major player in the Internet ecosystem seriously. User data and systems security is of highest importance and we are constantly working to keep up with the latest threats and best practices.

4) Internet Weirdness

The internet is a wonderful but strange place. As a service grows, it gets exposed to more of the stranger and harder to deal with things that happen on the web.

This problem gets confounded when you want to track metrics that reflect only the activity of real human beings (and exclude non-human agents).

Some examples of things that we have infrastructure in-place for and continually update to deal with at Bitly include:

    Bots

    Bots are computer programs that, to varying degrees of sophistication, emulate real user behavior on the internet.  Some bots are well behaved and are critical to the operation of the modern internet (e.g. web crawlers like Googlebot), others are more nefarious in purpose (scanners looking for vulnerable websites to exploit). At Bitly we have the tools in place to let the good bots do what they need to do, without impacting metrics, while blocking as many of the bad bots as possible.

    Facebook Prefetching

    Recently, Facebook has started to prefetch content that shows up in their users’ feeds on some of their platforms. Given Facebook’s scale, this can both result in a large amount of traffic and it can skew metrics if not properly accounted for. We’re actively working with Facebook to both make sure we have the capacity to give users the best experience possible while still accurately tracking user actions.

    Proxies

    A surprising portion of the world’s Internet traffic runs through proxies of various kinds. These proxies can create a variety of challenges to both accurately tracking metrics, mitigating abuse, and basic service delivery.  We regularly update our systems to properly track metrics from proxied traffic. Additionally, we regularly work with users and systems administrators to make sure that users operating behind proxies can reliably follow Bitlinks without issue.

    Misbehaving/Malicious Clients

    In a perfect world, everybody would follow the rules and implement their clients to respect standards and protocols. Unfortunately, we don’t live in a perfect world and there are all kinds of ways that clients can misbehave (intentionally or not): giant cookie headers, malicious/malformed content, spoofed values, slowed responses, syn floods, and more. If not handled properly, these bad clients can disrupt service and impact the experience for other users. Hardening systems to be resilient to such clients and dealing with new forms of abuse is an expensive and time consuming exercise. By operating at a large scale for many years, we’ve seen it all and are able to weather whatever misbehaving clients try to send our way.

    5) Scaling

    No matter how big or small your business, or whether you’re B2C or B2B, the goal for most is to grow.

    Even if you might not need the infrastructure to scale today, it’s not easy to add in if you do find that you need it somewhere down the line.

    Today’s consumer is connected to five devices, on average. This means you’re likely running marketing campaigns on multiple channels, of which might include SMS and email. There are over 77.5 billion consumer emails sent a day. This means a lot of links to process and keep track of.

    Since Bitly needs to support a large, active, customer base, and has for years, we already have enough capacity to more than handle the traffic that any one customer would generate.  Additionally, we are continually monitoring and modeling that capacity so we can stay ahead of the curve.

    6) Usability

    So we’ve talked a bit about data and resiliency. Layering these tools on top of your shortener means building your own user interface as well. How will you see how many clicks each link is getting? As the number of links you create continues to grow, how will you keep track of what you’ve shortened and when you shortened it?

    Building a simple web app for shortening is straightforward, but there are lots of “nice-to-haves” that either would be rather expensive to build on your own, or simply wouldn’t be possible, including:

    – A Chrome extension that allows you to do your shortening without leaving your other browser based workflows.
    – iOS and Android apps that leverage native sharing tools to allow you to use easily use Bitly as part of your mobile workflows.
    – Partnerships that let you use Bitly within all kinds of sharing, publishing, and analytics tools including Buffer, Adobe, Spredfast, and SumAll.

    7) New Features

    The link’s been around since the beginning of the internet, but it’s still evolving every day.

    One of the classic challenges of building something yourself is every time you want a new feature or to support a new standard, that doesn’t happen until you invest more time and money into your tool.

    Bitly continually adds new features that adapt with your needs and with the latest developments in the larger ecosystem.

    A prime example of this is mobile deep links. Deep links create a richer customer experience by driving users from the web browser directly into the app.

    We first shipped deep linking capabilities for users almost two years ago.

    At that time, the deep linking world was the wild west. Everybody was figuring out how things should work and there were no easy, off the shelf answers. Through a ton of research, testing, and development we were able to ship a feature that both enables linking directly into apps while still providing reliable metrics.

    Last year, Apple and Google both released deep standard protocols, Universal Links and Android App Links. These protocols make it easier for developers to build on mobile and make things a bit less wild west. But they also came with a number of challenges for developers, including making it much harder for marketers to get accurate analytics on their links. We’ve since made updates to our products and released native SDKs to make it easy for our customers to implement Universal Links and App Links.

    8) Network Effect

    Research shows that consumers trust friends and family, and even strangers, more than they trust brands. More than 8 in 10 people say that user-generated content from individuals they don’t know influencers what they buy, 51% even say that it’s more important than the opinions of friends and family.

    So not only is it important to understand what your customers are engaging with, but also what people in their extended network are talking about.

    A link shortener made strictly for internal use limits your ability to see past your immediate audience.

    At Bitly, it’s taken us eight years to build the Bitly Network to where it is today, and we’re still growing every day. The Bitly Network is a network of approximately 4 billion influencers and followers that are actively using Bitly. It encompasses all of the links that free users share – that’s a ton of metrics from links across various devices, channels, and locations.

    These engagement insights are especially valuable for brands when compared against content shared in their own network.

    For example, one well-known apparel brand found in their Bitly dashboard that while they had been promoting a lot of content around their wallets, the Bitly Network data was showing a lot of engagement around watches. Learning this, the team can then create a targeted campaign around the watches to nurture this specific group of customers.

    Seeing Across The Internet

    Building a link shortener is easy.

    But a shortener alone doesn’t cut it anymore. So, before you build a link shortener, the question to ask yourself, is this: how am I using the link to build the customer experience?

    By next year, all marketers expect customer experience to be their primary differentiator, according to a recent report by Gartner.

    Consumers are looking for a connection. Once you’ve caught their attention with a well branded link, you need to nurture that relationship by understanding where, why, when, and how they’re engaging.

    The true value of the link goes beyond branding. It lies in building, managing, and visualizing the customer experience.

WEBINAR

Check out how top brands use Bitly Enterprise for branding, analytics, multichannel campaign tracking, and more.

REGISTER!
Sean O'Conner @theseanoc
Director of Application Engineering