Identity, authentication, and authorisation from the ground up
In an earlier post we learnt the basic concepts of identity, authentication, and authorisation — important terms that are easily confused by hackers and technologists.
In this post we will dive deeper and demystify how apps actually implement authentication. Do it right, and you barely notice it. But do it wrong, and you lock users out or open major security holes.
We'll see how apps authenticate you, from old school passwords to slick new standards like WebAuthn, and we’ll get our hands dirty discussing security and usability trade-offs, encryption, individual sovereignty, and more!
By the end, you'll have a solid grasp of how real-world apps tackle authentication using the methods we've covered.
Let's get started!
Authentication 🪪
The first thing you do when you log into an app is identify yourself.
This might be using some kind of identifier — typically an email address or phone number — or logging in through Google, Facebook, or some other third party service, and using their knowledge of your identity instead.
Taking that identification and ensuring that it is yours is the process of authentication1. Once you’re authenticated, you can use the app!
In the earlier post we explained that authentication can be achieved in one of three different ways. You can demonstrate that you:
know something
own something, or
are something
Only the person you're identifying as would know or own this information, or have that identity.
All modern apps use a technique that boils down to one of these three methods to authenticate you.
So let’s look at each in turn!
Knowledge-based authentication 🧠
Password-based authentication uses the first of these authentication methods — it verifies that you know something only the user you're logging in as would know: their password, pin, passphrase, or any other secret shared between the user and the service.
To prevent attackers being able to access your account in case an app’s authentication data is leaked through a hack, the app doesn’t actually store the password, but a hashed2 version of it.
Roughly speaking, the hash of your password is like an encrypted version of your password, except that given your password’s hash, it’s very hard (near-impossible, if hashed correctly) to retrieve the original password3.
Limitations of passwords
Implementing password-based authentication yourself is fraught with difficulty.
Some of the “fun” challenges you can anticipate are:
Properly implementing passwords — You need to manage salting, hashing, encryption and then store them securely in your database.
Password recovery flows — You’ll also need to implement email-based password recovery. But what if your password-recovery emails go to spam? Having angry users contact customer support because they can't log in is not fun.
Handle "password1@3" — You'll quickly realise that people like to use weak and easy-to-guess passwords.
This could allow hackers to find a password through a dictionary attack4 — something they can do if they get access to a leaked version of the database, for example — before later gaining access to the running system.
Block bots — Users also really like to reuse passwords5, so you might have hackers attacking your system because of a data breach on a completely different website. Blocking these hackers when they’re attacking your site from many different IP addresses is a major challenge6.
Additional friction - There will be quite a few users who won't have the patience to go through the password-recovery flows after forgetting their password. Instead, they just give up and go do something else. Forcing your user to remember something to log-in can add additional friction and increase churn.
And much more7
Each of these steps is hard and requires scarce engineering resources, which is why authentication providers such as Supabase, Auth0 and Clerk offer managed versions, at a cost.
But most significantly, users may forget their password and never even use your site in the first place!
To summarise: passwords are a poor authentication mechanism. They’re hard to implement, unsafe, and lead to a poor user experience!
So: is there a way to authenticate users without relying on something that only they would know?
I wouldn’t still be writing if there wasn’t, so let’s get into that!
Ownership-based authentication 🔒
Ownership-based authentication is our second authentication method — it involves verifying that you own something that only the user you are attempting to log in as would own, such as their email inbox or phone number.
One-time passwords
For instance, when you log in, an app might send you a one-time password (OTP8) via email. This is typically a random short, server-generated, alphanumeric code like ZXY1, but it can also be a magic link, which is a URL containing the OTP as part of it.
The OTP is a temporary secret that is only shared between the app and the user's email inbox, is valid for that specific interaction, and is invalidated once used.
If you log in using this OTP, then you’ve successfully demonstrated ownership of the email address connected to the user account, and so you must be that user — you’re successfully authenticated!
SMS-based logins can also be used, but SMS are not encrypted and can easily be discovered by anyone on the network by sniffing for text messages9.
SMS-based logins can also be gamed for profit — a big reason why many apps no longer solely rely on SMS for authentication, or even only offer SMS based authentication for premium or paying users.
But worst of all, SMS and emails may take minutes to be delivered, or may never be delivered at all10!
Is there a way for the user and service to coordinate on a shared, time-based secret, without sending it over an insecure, slow and unreliable communication channel?
I haven’t stopped writing, so yes!
Time-based one-time passwords
Authentication apps, such as Google Authenticator, provide time-based OTPs (TOTP11).
These authentication apps are set up well in advance of logging in, typically during the first onboarding session. Instead of using SMS or email to send the OTP to the user, an app on the user’s device generates a TOTP using the current time and a secret shared between the user and the web app.
This is much safer than SMS or email OTP delivery, as it does not rely on the security of external communication channels. The TOTP is generated locally on a device the user possesses and controls, making phishing and interception attacks much more difficult.
Hardware authenticators
Finally, hardware security keys, such as a YubiKey12, are a physical authentication device that a user can carry around with them. These are often used in high-security environments, such as in an enterprise, and can also create OTPs. These are really powerful devices, and we’ll dig into them more soon.
Whilst ownership-based authentication methods are often more secure, easier to use (no need to remember a password!), and easier to implement than passwords, they still require the user to do something cumbersome to authenticate.
This is an opportunity to switch device, get distracted, churn, and never use your app at all!
For this reason, authentication methods that require the user to switch device are used sparingly, only for sensitive operations such as when logging in on a new device for the first time.
So — can we authenticate a user without requiring them to use two devices? Absolutely!
WebAuthn 🕸️
WebAuthn13 is an awesome, new(ish) protocol that allows users to authenticate themselves whilst browsing the web.
Instead of using emails, passwords, YubiKeys, or OTPs, the information needed to authenticate a user is stored in an authenticator — typically the browser or device.
Public-key cryptography
The method used for authentication is public key cryptography. It’s the same technology used in Transport Layer Security (TLS) — a networking protocol which is used for nearly all traffic on the internet. It’s fundamental to security, so let’s take a quick aside to dig in:
The idea behind public-key cryptography, or asymmetric encryption, is pretty simple: each person has two keys — a public and private key. The public key is shared with everyone, and the private key is kept to themselves.
Seems pretty simple so far. But we can now do two magical things:
Anyone can send the user a message by encrypting it with the user’s public key. Only the owner of the private key — the user — is able to decrypt that message.
The user can use their private key to sign a message. Anyone with the public key can verify that that user signed the message, and no one other than the user is able to sign it as that user.
Public-key cryptography in WebAuthn
The browser or device, acting as an authenticator, generates a new public/private key pair as the user onboards onto a web service. The private key is kept hidden, potentially stored securely in a hardware security module (HSM14) on the device, and the public key is shared with the web service.
When the user later wishes to log in, the website asks the browser to sign a specific message. The browser or device does so using the private key, the website verifies it is indeed the same user using the previously shared public key, and the user is now authenticated!
https://webauthn.io has an awesome demo of this — try it out!
With WebAuthn, the user doesn’t need to type their username or remember a password and can login with a single click — very impressive!
And excitingly, the prompt to log in with WebAuthn will only appear on the correct site, thereby preventing the phishing attacks that were possible with passwords 🎉.
But logging in with a single click also makes it easy for friends, enemies, and attackers to log in with a single click, if they got close enough to your machine.
So, can we ensure it’s you actually doing the clicking? Surely!
Multi-factor authentication
The HSM can actually prompt the browser for something before it allows access to cryptographic operations, such as generating a public and private key or signing a message. And the browser can prompt the user for this data in turn.
The prompt can ask for a password — another example of knowledge-based authentication — or, even better, ask for biometrics, such as your fingerprint or face. Laptops and mobile phones commonly support fingerprint readers, so this is a fast, easy, and secure15 way for users to authenticate.
Clicking a button to sign in and using your fingerprint is multi-factor authentication, since it requires two forms of authentication for you to log in: ownership of the device with the HSM and your inherent identity and ownership of your fingerprint.
And because you attest to your own identity, this is a form of self-sovereign identity — we’ll talk about this more in the next post.
Cross-device authentication
WebAuthn does pose a problem, though.
Users typically use a website on multiple devices: their phone, tablet, and laptop. It’s easy to share a password between devices — does WebAuthn offer something similar?
Yes! Well, sort of.
You can delegate the authentication to a roaming device — which could be your phone, laptop, or YubiKey — anything that is accessible via bluetooth, NFC, or USB. In turn, that device can sign the message on behalf of the main device.
This is an additional layer of protection, since malware on a site would need to infect an entirely different device than the main one you’re using to steal your private key — neat!
However, this means that we’d always be reliant on that first device for logging into that service, or we would need to set up a private key on each device we want to log in from — hardly a good user experience, and potentially worse than multi-factor login using passwords and OTPs.
Limitations of WebAuthn
WebAuthn isn’t a panacea, and does come with some notable downsides:
Single point of failure — If you lose a device, you’d lose access to all of your logins that that device stores the private keys for. Recreating those logins would take ages, and would be a massive pain!
Passkeys16 are an exciting new feature to help plug this gap and to share private keys across devices, allowing for ease-of-access and easy recovery. The fun part is that Google17 and Apple’s iCloud would sync these private keys for you — it’s built into the platform!
As excited as I am about Passkeys, they are new and controversial1819, and it’s unclear if they’ll achieve mass adoption.
Inflexibility — Passwords are flexible enough to support most flows that users need20. In contrast, WebAuthn only supports a subset of flows.
For example, sharing access to a single account (eg to access the bills of an energy provider) with a friend is straight-forward with passwords, but much tricker with WebAuthn.
Uncertain adoption — Perhaps most important is that users aren’t familiar with the passwordless world.
Will they onboard? Will they log in again on a new device? Will they understand the user flow? We don’t know yet — and it’s a risk to try out something as innovative as this.
Whilst I really want a passwordless future, and think WebAuthn and Passkeys could be our saviour, there’s one more form of authentication that is widely used and worth learning about.
Identity-based authentication 🙋
The third way we can authenticate is to use something we are — our identity. Biometric authentication is one example of this, and we briefly covered it in the last section.
But we can also implement this by outsourcing identity verification to a third party that users already have an account with — an identity provider21.
When you see “Login with Google” or “Login with Facebook”, that's identity-based authentication in action.
The identity provider tells the app that you are indeed the person you claim to be by providing it with a signed ID token.
When the app you’re signing into gets this token, it uses the signature to verify that the identity provider did indeed create that token — and now you’re fully authenticated!
OpenID Connect
OpenID Connect (OIDC22) is the underlying protocol used for this flow. OIDC is a layer on top of OAuth 2.0 — a protocol used for access delegation23 — that standardises identity data like your name and email address.
Big tech giants like Google, Facebook, Twitter, and Microsoft all offer identity services that implement OIDC. It's easy and convenient for users since they can leverage their existing accounts.
And it saves a ton of headaches for developers since they no longer need to build secure authentication systems. Instead of authenticating a user yourself, just plug into Google, and you're done!
This is an example of federated identity24, where a user’s identity is used across multiple systems — Google, Facebook, and all the apps that rely on these identity providers — instead of just one.
Limitations of identity federation
However, identity federation has a fundamental flaw: the identity provider can revoke access at any time, instantly locking you out of all sites. You don’t own your identity — it’s stored on Google’s servers, not yours.
You may think it’s rare, but it’s surprisingly common2526 to find posts on Hacker News of users being denied access to their Google or Facebook accounts due to one reason or another, and totally unable to access any service that they used Google or Facebook as an identity provider for.
The quest for true user-owned identity is an active area of innovation which I'll cover in an upcoming post! Subscribe now to be updated when it’s released 👇
I would love to hear folks thoughts on this — feel free to comment below or DM me!
Have a great week all — stay tuned for Part 3!
Authentication is in fact broader than simply validating your identity — it can refer to proving any assertion, such as your age being greater than some threshold, that you have a valid driving licence, or that you have enough funds in your bank.
We can make the hash even harder to crack by adding salt to the original — random data that prevents attackers using pre-computed data to guess your original password.
About 64% of people reuse at least one password across multiple services: https://auth0.com/blog/what-is-credential-stuffing.
There’s a few other reasons why passwords are challenging — and many of these are outside of your control as an app developer. Malware can log the keys the user presses whilst typing the password to secretly exfiltrate it, an enemy (or friend!) could watch over your shoulder as you type it in (this is known as shoulder surfing), and phishing attacks could prompt the user to enter their password when they’re on similarly branded but malicious website.
Your biometrics are stored securely, and never leave your device — they’re only used to unlock the HSM used to store your private keys.