@theosteiner.de talked with @danabra.mov about the AT Protocol. Rather than a general overview, they walked up the stack layer by layer — comparing it with HTTP at every step — from AT URIs and records to DIDs, personal data servers, lexicons, and relays.
This is an English episode.
Guest
Dan Abramov (@danabra.mov)
Previously worked on the React team, then at Bluesky
Now works part-time on the Next.js team at Vercel, and plans to relocate to Japan
Writes about ATProto on his blog, Overreacted
Walking up the ATProto stack
What ATProto is
“AT” officially stands for Authenticated Transfer, but the name also echoes the @ in usernames, because the protocol is fundamentally about identity
A good protocol stays invisible. ATProto is a new way to build social apps, originally designed at Bluesky — and originally intended for Twitter itself
Today, what you create stays trapped inside the app that hosts it: an answer on Stack Overflow, a comment on GitHub. ATProto lifts that content out into an app-agnostic layer where everything behaves more like files
When you post on Bluesky, you are really writing JSON into a folder in the cloud that you control. Anyone can build a competitor with all the existing data already present and interoperable — Blacksky is a real example, with its own moderation policies and no use of the Bluesky API
Because every app writes to the same layer, app authors do not need to provide an API. The protocol is the API
Layer 0: what it builds on
ATProto does not reinvent the internet. It uses HTTP and WebSockets for transport, CBOR for encoding, DNS for resolution, and public-key cryptography for signatures
Where existing primitives fit, it reuses them: domains are used as handles rather than inventing a new naming authority
Where they did not fit, it invented: Lexicon, its own JSON type-definition system, chosen over RDF, JSON-LD, and JSON Schema
AT URIs and records
An HTTP URI points at a page or a file. Every AT URI points at a record — a piece of JSON with a typed schema. A Bluesky post, a follow, a star on Tangled: all records
Each user owns a repo, and records live in the repo of whoever created them. If a million people follow you, that is a million follow records in their repos, not yours. If you follow a million people, those records are yours
Records link to each other, which is where AT URIs come in: a comment in your repo links to a post in someone else’s
The authority part of an AT URI is not a physical host but the user. That indirection is the whole point — it lets you move your hosting without breaking a single link
The friendly form uses your handle, which can change. The canonical form starts with a DID, which never does
Dan’s framing: with HTTP the authority is the location; with ATProto the authority is the person. It is closer to RSS and Google Reader — you control the feed, and apps are just clients that read it
Identity: DIDs and the PLC directory
A DID is a W3C standard and effectively a meta-standard: did: followed by a method that specifies how to resolve it
ATProto supports two methods. did:plc (originally “placeholder”, now “Public Ledger of Credentials”) is centralized and run through a directory; did:web resolves through a normal domain, but if you lose the domain your identity dies with it
Either way, resolution yields your DID document — think of it as a passport with three things: where your data is currently hosted, your handle, and your public key
The PLC directory is auditable rather than trustless-by-decentralization: fully open source, no private state, replayable from public data, with a queryable history for every DID
Each change to a DID document is signed by the key from the previous version, and the first version’s hash is your identifier. So the entire chain of changes to your hosting, keys, and handle can be verified independently
Hosting: the PDS
Your records live on a Personal Data Server. One PDS can serve many users, but your own source of truth lives in exactly one place
A PDS serves your JSON and offers a WebSocket connection so apps are notified of changes in real time, instead of polling
Migration works like changing a Git remote: the repo is copied to the new server, media is moved (still a rough edge — it currently goes server-to-client-to-server), and finally you submit a change to the PLC directory pointing your DID document at the new host
Apps barely notice, because AT URIs do not change and most apps have already cached the data
Theo migrated his own account from the default Bluesky hosting to eurosky.social
PDSls is a good way to see this for yourself: enter a handle and it resolves the DID, resolves the host, and shows the whole repo as collections of JSON
Lexicons
Collections are grouped by record type, named in reverse-domain form (app.bsky.feed.like), which gives every app developer a namespace — much like scoped npm packages
Records carry a $type field naming their collection, so a record is self-describing even without its URI
There are two kinds of links: a weak link is just an AT URI; a strong link adds a CID, the hash of the linked record, so an app can tell whether the target changed since it was referenced
Lexicons are the schemas behind these types. Codegen tools turn them into parsers and validators for JavaScript, Swift, Go, and more
You do not have to publish a lexicon — but if you do, you publish it as a record. PDSls resolves published types and renders them like documentation
You can also reuse other apps’ formats. Three blogging platforms — Leaflet, Offprint, and pckt.blog — converged on a shared lexicon called standard.site, and the Bluesky app now reads it to give those links special treatment with the publication’s brand color
Teal FM shows how far this goes: the app has not launched, but the lexicon exists, so people already write listening records and third-party apps already display stats from them
Dan’s framing: if you had the world’s JSON, why not run joins across products? Top tracks among the people you follow, trending repos among your friends. Apps become lenses over a shared web of data
Relays and building apps
Apps do not hammer every PDS. Relays retransmit a stream of commits and identity events over WebSocket, so an app can subscribe once to everything on the network — or filter down to the collections it cares about
The stream carries signed commits, so you do not need to trust the relay. That is the “authenticated” in Authenticated Transfer. Running one is reportedly around $30 a month at current volume
If your app defines new record types, no backfill is needed — connect to a relay, filter, and you will see every record from the moment it is created
For existing data there are two routes. TAP synchronizes the collections you name and then follows new events. Or use Constellation, a community-run backlink index: given a record, it returns everything that links to it, which is enough to build a working app with no backend at all
Because you should shield PDSs from your app’s traffic anyway, load naturally concentrates on the app layer rather than on hosting. Dan’s view is that apps — CDNs, video transcoding — are far more expensive to run than serving JSON
Auth collapses too: login uses OAuth, but the OAuth provider is your own PDS. After migrating, Theo’s login screen is eurosky.social
Reliability is an early-adopter story. Upstream problems do surface, and the answer is to run your own relay if you need isolation. Dan compares the moment to open source in the 90s
What’s still missing
The biggest gap is permissioned data. ATProto deliberately started with public data, because scaling down from public aggregation is easier than scaling up from private
Work is underway on a permissioned-data extension for things gated by membership, such as groups. End-to-end encrypted messaging is a separate problem
Until then, private and group state has to live in your own database. Roomy Chat, a Discord-like app on ATProto, is one of the projects waiting on this
Svelte is notably popular in the ATProto community. Dan’s guess is aesthetic affinity: both feel webby
Where to start
Try Bluesky for real, and replace the default Discover feed with the community-run For You feed — findable from the Feeds tab. It learns from your likes, and it runs off someone’s gaming PC at home, which is possible because a feed is just an HTTP service returning an array of AT URIs
Starter Packs let you follow everyone in an area of interest with one click
@TheoSteiner So, I'm not used to doing this introduction in English because usually this podcast is in Japanese. So, bear with me if it's a lot of ums and errs, but this is the UIT Inside podcast. UIT is the front-end organization at LY Corporation, which is a Japanese company, and our major products are the LINE Messenger, which is the leading messenger in Japan, and Yahoo Japan, which is the Japanese version of the Yahoo search engine. And today I'm really excited because we have the most amazing guest ever on. We have Dan Abramov on the podcast. Dan, can you introduce yourself?
@danabra.mov Hi, thank you. My name is Dan. I guess I used to work on React, so that's how you might have seen my avatar on GitHub sometimes. And then I used to work at Bluesky. And now I'm working part-time on the Next.js team at Vercel.
@TheoSteiner And you work for Vercel Japan, is that right?
@TheoSteiner そして今はVercel Japanで働いているんですよね？
@danabra.mov That's the plan. So currently I'm still in the UK, but I submitted the visa documents. So we'll see how that comes through, but the plan is to relocate to Japan, possibly at the end of the—
@TheoSteiner Yeah. And obviously I had you introduce yourself, but you don't really need an introduction. When I told my colleagues that you were going to come on the show, they were all so excited. So, you're not just famous in the English-speaking world, but every Japanese front-end developer also knows you.
@danabra.mov Oh, interesting. Because I thought people already kind of... because I disappeared a little bit from the scene. I think there's a new generation who... I haven't really talked too much.
@TheoSteiner Yeah, I guess there's a new generation as well, but you're still the legend, I guess.
@TheoSteiner たしかに新しい世代もいますけど、やっぱりDanはレジェンドですよ。
@danabra.mov Yeah.
@danabra.mov そうなんですね。
@TheoSteiner So we had a meeting last Thursday, and I was like, "Yeah, Dan is coming on." And they all spammed the chat with emojis because they were so excited. And I've never had that happen with a guest coming on before.
@TheoSteiner But today we're not talking about React. We're actually here to talk about something completely different, which is the ATProto protocol. So, ATProto—what does ATProto even stand for?
@danabra.mov Ah, ATProto. It's like "AT Protocol," and "AT" supposedly stands for "Authenticated Transfer." So I think that's kind of the formal name. I think really the reason is because it looks like the @ symbol that you use in usernames, and it's a protocol that has to do with identity. So it's kind of like when you say my Instagram handle is @whatever, except it's an internet protocol instead of something specific to a company.
@TheoSteiner Okay. So you and I are both on Bluesky, so I think we have quite a bit of contact with the AT Protocol. But the majority of Japanese developers are still stuck on Twitter or X, and they might have never heard of AT Protocol.
@TheoSteiner Can you just give us the 100-foot bird's-eye view of the protocol? What is that?
@TheoSteiner じゃあ、そのプロトコルについてざっくり説明してもらえる？どんなものなの？
4. A bird's-eye view: lifting your data out of apps
@danabra.mov Yeah, it's very tricky to answer because a good protocol stays invisible. It's a way to build social apps—not necessarily social, but I guess that's an easier area to focus on. It's a new way to build social apps that was originally developed by Bluesky, the company that you might know as a Twitter clone product. But really, originally they were supposed to do that with Twitter itself. It's just that Twitter broke the contract with them. But the thing they originally wanted to build is a generic, universal internet protocol. I've tried to describe it in different ways. I don't know which one resonates. Different audiences understand it differently. One way I would describe it to web developers is that it tries to extract an identity layer for the web. So each app has its own login, and for social apps you can create different kinds of things in the app, but they stay within that app, right? Like if I write an answer on Stack Overflow, it's kind of stuck inside Stack Overflow. If I write a comment on GitHub, it's kind of stuck inside GitHub, in the sense that these companies own what I made using their products. And then if these companies start changing in ways that you don't like, maybe you can abandon them, but you've invested so much effort and so much content into these closed spaces that they control. And the idea of AT Protocol, or shorter, ATProto, is to kind of lift that out so that everything works more like files. So when I post on Bluesky, I'm not actually putting stuff into the Bluesky app. Although it looks like this, right? If you just use it, it kind of looks like Twitter. Or if you use Tangled, which is ATProto's GitHub alternative, it kind of looks like GitHub. But what happens behind the scenes is actually it's almost like I have a folder somewhere in the cloud that has data from all of these apps as JSON. So everything is stored as JSON in this new layer of the web that is app agnostic. It's not controlled by one company necessarily, right? You can host this folder with different providers. You can host it yourself. It is really cheap. But it's app agnostic, and as a result anyone can start a competitor to one of these services or one of these apps with all existing data already there and already interoperable with existing apps. So if somebody wants to start a competitor to Bluesky—and this actually happened—there's Blacksky, which has different moderation policies and so on, but it's completely interoperable with the Bluesky app even though it doesn't use the Bluesky API. So that is the big thing. Every app on ATProto shares a common API where you don't need to provide an API as the app author because all apps just write to this JSON. It's a common way for apps to, in a sense, talk to each other—or show content from each other and augment each other—without actually talking to each other. So without exposing an API, because the protocol itself is the API. There's a few other ways I like to explain it, but I think that's the big idea: freeing the stuff that we create, lifting it out of apps, and making it live on an independent layer that we actually control as users, so that everything we create is more like files under the hood, but with the same normal user experience for the app user.
@TheoSteiner And you actually have a great blog post on this. You have "A Social File System" on your blog, Overreacted, and it talks about what you just did the perfect summary of.
@TheoSteiner しかもこの話、あなたのブログ「Overreacted」の記事『A Social File System』でもすごく分かりやすくまとめてるよね。
5. Walking up the stack: what ATProto builds on
@danabra.mov Yeah.
@danabra.mov そうだね。
@TheoSteiner Actually, we can just end the podcast here. But I think we want to dive a bit deeper, and since this podcast is mostly for web developers, I actually prepared a little ambush for you. So, ATProto is a protocol, and web developers are usually familiar with the HTTP protocol. So I sort of wanted to go up the stack layer by layer and see where they differ, what ATProto offers on top of HTTP, and how they even build on each other.
@TheoSteiner And this might feel like a check of your HTTP knowledge, but we'll keep it real simple and easy. So, layer zero of HTTP is: what is HTTP even for? It would be a communication standard between clients and servers. And you already hinted at this, but what is layer zero behind the ATProto protocol?
@danabra.mov Yeah. ATProto kind of builds on top of multiple things. It builds on top of HTTP, DNS, JSON, WebSockets, CBOR, and I think maybe these are the ones that I know. I'm sure there are other things, but it's HTTP or WebSockets as the transport, and then CBOR as encoding, I think, for JSON, and then DNS for a bunch of...
@danabra.mov Yeah. And I guess it has a few of its own inventions. There's something called Lexicon, which is a JSON type definition. They did not go with RDF or JSON-LD or JSON Schema. They made their own because none of these things quite fit what they wanted. So there are some custom pieces, but where appropriate it tries to reuse existing primitives. For example, domains as handles is one example. Instead of inventing its own handle system, it's like, well, it has to be centralized in some sense, because somebody has to determine the top-level mappings, like who owns this username. And they decided, well, this already exists on the internet. It's called domains. So we're just going to use this existing system for this. I think there's a lot of that in the design. And then there's also cryptography. In addition to all of this stuff, there's cryptography—not in the sense of crypto, blockchain, or whatever, even though there is one element that's maybe reminiscent of that. We can get to that if we want to. But overall it's much more like the web. There are cryptographic signatures and public-key/private-key cryptography.
6. AT URIs, records, and whose repo holds a follow
@TheoSteiner So whereas HTTP—HTTP and ATProto both have URIs. For HTTP, everyone in the web world knows URLs. You have a domain that maps to a server, and DNS helps resolve the path to the server. But in the ATProto world, what is the common URI? What is in there?
@danabra.mov So in ATProto, all AT URIs... First, it's worth talking about the AT URI itself. It's `at://`, and then what goes after it? I think the first question is: what does it point to? An HTTP URI points at a song, an image, an HTML page, or something like this. Maybe that HTML page has more links. In ATProto, all AT URIs point at records. And a record is a piece of JSON with a typed schema. Essentially, every AT URI points at some JSON. For example, a Bluesky post is a piece of JSON. Or Tangled—the GitHub alternative—a star is also a piece of JSON. A Bluesky follow is a piece of JSON that says you follow this person. So it's the stuff that you create: these documents, or records, through the apps. These records can link to each other. For example, if I comment on your post, each of us has a folder, or a repo, where all of our stuff lives. The important part is that you have yours and I have mine. Each of us owns our own data. Sure, maybe we use some hosting behind the scenes. When I sign up for Bluesky, I don't think about where it lives. But I actually moved my hosting, so you can move it over time. All my stuff is in my repo, and all your stuff is in your repo. Actually, I'm going to give you a test to see if you understand it. If 10,000 people follow you, where do these follow records live? Are they in your repo, or are they somewhere else?
@TheoSteiner I would say they are distributed across 10,000 users, right?
@TheoSteiner それぞれ1万人のユーザーに分散されている、って感じですよね？
@danabra.mov Yeah.
@danabra.mov そうですね。
@TheoSteiner Because everyone holds their own follow.
@TheoSteiner みんな自分のフォロー情報を持ってるからです。
@danabra.mov Yeah, exactly. So if you have a million followers, they're not actually yours. There's one follow record per person who follows you. But if you follow a million people, that's a million records in your repo, because it was your choice to follow them. So now that we understand your stuff lives in your repo and my stuff lives in mine, if I comment on your posts, that's a comment record in my repo that has a link to posts in your repo. And that's where AT URIs come in. I need a way to link to something that exists in your repo—to some piece of JSON. So we have this web of JSON where, instead of HTML pages with links, we have JSON with links. That's where the need for a separate protocol comes up. If we just used HTTP, imagine there's some server hosting your data at `hosting-company.com/you/post`, and I link to that from my repo. That means you can never change the hosting company without breaking the link. We have the domain system, which lets whoever hosts the content change where it's hosted—change the IP address—but it doesn't let you move away from that hosting company without breaking the links, because it's still hosted by `hosting-company.com`. From a product-design perspective, that is the big reason why everything went wrong. We put stuff into these apps, into these boxes, and then it stays there because they have power over us. So coming back to your question: what is in the AT Protocol link? It also starts with a domain authority. You may see something like `at://dabra.ms/...`. But the important part is that the first thing is not where it's hosted. It's not the physical server. It's not the domain for the physical server. It's my username, in a sense. So there is an extra resolution step where we find out, okay, for this username, where is the current server where this is hosted? Then it goes to HTTP to fetch it. So it's one indirection that puts usernames before the physical storage. That lets me change where my actual hosting is over time, and none of the links will break. That's the big motivation for why it's done this way.
@TheoSteiner So if I rephrase this in my own words: the issue with HTTP is that it uses URLs, and URLs point at locations. The issue is that those locations can change—they're sort of ephemeral. But ATProto says we somehow need this to never change. We need some resource identifier that will always point at your repo.
@danabra.mov Yeah. Or you could say that it literally puts people first. In a URL, the authority part is the physical host of the location. In an AT URI—and there's a whole tangent about whether it should technically be called a URI—but essentially the authority part is the user themselves. So it's not like `bluesky.com/danabra...`; it's `dabra.ms/...`. It inverts the relationship so that the user comes first. There's one nuance I wanted to flag. When we talk about the username, that's the friendly way to write an AT URI. But it's actually unstable because you can change your username over time. The full form of the URI starts with something called a DID, which is a permanent identifier that never changes. So this is the thing that's forever immutable, as long as you're in control of your identity.
@TheoSteiner Okay, I want to come back to the DID part because I think it's really important. I didn't know it was pronounced "did"—no pun intended. So to get this back to the HTTP discussion: in HTTP you have a domain. You could perhaps lose access to your domain, and the domain sort of allows you to obfuscate where your data actually lives. But you could lose the domain, and that's the issue. And to solve that, the AT Protocol adds a different layer that says, "No, the domain, or your server, is not what we're looking for. The identity is what we're looking for. And if we have your identity, we can resolve that to where you host in HTTP."
@danabra.mov Yes. I wouldn't say it's because you can lose it, although certainly you can lose a domain. I think it's more about... if we're talking about something like a tweet, it's not going to live on your domain anyway. It lives on Twitter. So if we try to design this thing with HTTP, then the HTTP URI would be something like `twitter.com` or `twitter-data-storage.com`, or whatever. But then the problem is you can't walk away from Twitter and use a different app. You can't get rid of Twitter without breaking all these links. It's more about how, when each of us had our own site, we were directly connected to how the internet works. If we don't like the hosting, we just change our hosting. Our users don't need to know—it still resolves. But when we moved into social spaces like Facebook, Twitter, or Stack Overflow, we're now one step disconnected from how the internet actually works because we can't change our hosting. You can't change your hosting when you're using Twitter. Twitter couples the app and the hosting. And so the idea here is: let's still make your content live in a different world where you can change the hosting, it's app agnostic, and the app is just one of the consumers of this layer of the internet. In that sense, I like to think about it as similar to RSS and Google Reader. You control your RSS. You can move where you host it, and then you can consume all of the RSS you're interested in from apps such as Google Reader. But they actually have no sway on the underlying system. If Google Reader disappears, somebody else can make an RSS client that reads from all of them. So it's really about avoiding lock-in. It's about removing the power from the apps to keep you locked down.
@TheoSteiner That's a really nice way of putting it. And now that we know the promise of the AT Protocol, I kind of want to dive into how it works. So you said there's this DID that can be resolved to your identity. For HTTP, the domain would be resolved to your location via DNS. You do a DNS lookup and you get the IP address. How is the DID resolved to you?
@danabra.mov Yeah. So let's first start with what a DID is. This is a W3C standard. They tried to reuse this part. It's kind of a meta-standard in the sense that it doesn't actually specify how to resolve a DID. It's more like a DID is a format where first it says `did:`, and then there's something called the DID method. This can be three letters—or maybe more—but the method specifies how to find a certain thing: a piece of JSON that's your identity. From that point, the rest is just an identifier for this method. So it's almost like DID is just a universal umbrella for many identity systems. ATProto picks two DID systems supported by the DID standard because they satisfy certain properties. One property is that they have to be fast to resolve. Something like "download the whole blockchain" is already a non-starter. It has to be something that's easily resolvable. I don't remember the exact properties, but they basically picked two things. One is the one they invented. There are two DID methods supported by ATProto. One is called `did:plc`, which initially stood for "placeholder" because they thought they were going to replace it. Then they decided, "Actually, it's quite good. We're going to keep it." Now it's supposedly "Public Ledger of Credentials."
@danabra.mov そうだね。まずDIDが何かから説明しようか。これはW3Cの標準で、その部分を再利用しようとしてるんだ。DIDは、実際にどうやって解決するかまでは決めていない、いわばメタ標準なんだよ。DIDは、まず`did:`というフォーマットがあって、その後にDIDメソッドというものが続く感じです。このメソッドは3文字だったりもっと長かったりしますが、どのようにして自分のアイデンティティとなるJSONを見つけるかを指定します。その後ろは、このメソッド用の識別子になっています。つまり、DIDは色んなアイデンティティシステムをまとめる共通の仕組みみたいなものですね。ATProtoは、DID標準でサポートされている2つのDIDシステムを選んでいます。これは特定の条件を満たしているからです。その条件の一つは、解決が速いことです。例えば「ブロックチェーン全体をダウンロードする」とかは論外ですね。簡単に解決できる仕組みである必要があります。正確な条件は覚えていませんが、基本的に2つを選んだということです。そのうちの一つは、彼らが作ったものです。ATProtoがサポートしているDIDメソッドは2つあります。一つは`did:plc`で、最初は「placeholder」の略でした。最初は置き換えるつもりだったけど、「意外と良いじゃん」となってそのまま使うことにしたそうです。今は「Public Ledger of Credentials」という意味になっています。
@TheoSteiner Okay, that's a crypto-like ledger.
@TheoSteiner なるほど、それは暗号系の台帳っぽいですね。
@danabra.mov Yeah. I think that is an influence from that world. But it's actually not distributed. This is the centralized part. It's kind of like an npm registry. The other DID method is `did:web`, which uses a normal domain lookup. So these are two ways this can work. In both cases, the goal is to resolve to a piece of JSON that is called your DID document, which I kind of like to think of as a passport. It's a piece of JSON that says three things. One is where your stuff currently lives—your HTTP hosting. Another is your handle—your domain username. It can be completely different from where it's hosted. The third thing is your public key. So if somebody wants to verify some record, what is the public key that they verify against? These are the two methods. The difference between them is that one is more centralized. Legally, they're spinning it off into an independent Swiss organization. The goal is for it to evolve similarly to how the root DNS has evolved with ICANN, with some neutral organization managing it. Being centralized is kind of the point, because that is the authoritative place to say what's under this identity. But they came up with a bunch of things that make it less of a problem that it's centralized. For example, it's entirely open source. There is no private state in it. It's fully replayable based on public data. You can audit it. There's an audit log. You can inspect every transaction. It's highly auditable. That's the usual way. If you sign up on Bluesky, you're going to get a `did:plc` identity. PLC is the thing that holds your DID document. But if you hate the idea of anything centralized, there's `did:web`, where the identifier is literally a domain instead of a hash. With `did:plc` you get a hash. With `did:web` you get a domain. But if you lose the domain, you can't control your identity anymore—it effectively dies. So that's kind of the choice for hardcore decentralizers.
@TheoSteiner So the PLC directory is a centralized place to look up DIDs that use the PLC method. And "Public Ledger" sort of implies that there's a way I can look up the ledger. I can see what happened on the ledger, and "public" means probably everyone can look it up. Can I just do an HTTP request somewhere and get information about my ID?
@danabra.mov Yeah. There is something called the PLC directory. It's a website. It has an example where DID resolution can be as simple as: `curl https://plc.directory/<did>` and that gives you your JSON. So there's an API. There's also an API for retrieving past versions. You can see, for every DID document, what it says now and what it said in the past. You can verify that each change is signed by the public key from the previous version of the DID document. So you can independently verify that the DID document could not have been tampered with, because it's signed by the user's private key, which the PLC directory doesn't have. And then the first version of the DID document, when hashed, that hash is your identifier on PLC. So you can verify that the first version is also correct because it has the signature. It's a hash of the signed version of the first document. So you can recursively verify, for every user, the entire chain of changes to their identity, including changes to hosting, the key pair, and the username.
@TheoSteiner So it's a cryptographically verifiable chain of events of what happened to your passport, so to say, with no loose ends, because the first entry is the DID itself—the identifier itself.
@TheoSteiner Very cool. And you said there are three parts living in there.
@TheoSteiner すごいですね。それで、中には3つの要素があるって言ってましたよね。
@danabra.mov Yeah.
@danabra.mov そうです。
@TheoSteiner There's your alias, there's where your data lives...
@TheoSteiner エイリアスがあって、データの保存場所があって…
@danabra.mov Mhm.
@danabra.mov うん。
@TheoSteiner ...and then what was the last part?
@TheoSteiner …で、最後の要素は何でしたっけ？
@danabra.mov The public key.
@danabra.mov 公開鍵です。
@TheoSteiner The public key. Okay. So people can verify that it's actually what it claims to be. So where does the data actually live? What would be in my DID document?
@danabra.mov So it depends on how you made your ATProto account. We call this layer above the web the "atmosphere." So having an atmosphere account is kind of like having a Google account, except it's the web. I think that's part of the idea—that we all just have these web accounts instead of many accounts tied to a specific provider like Google. If you follow the DID document, it will say... For example, if I sign up for ATProto through the Bluesky app, it has to create hosting for me because I don't already have ATProto hosting. Bluesky is actually two different things. There's Bluesky hosting under the hood, and there's the Bluesky app. They're completely independent. I actually changed my hosting to a different provider. You can also self-host. You can run a server yourself. It's really cheap. If you sign up through Bluesky, it's going to point to something like `*.host.bsky.network` or whatever. This is the server that hosts your JSON, contains your private key, and is able to serve your data. If apps want to know what you've posted, it can serve your posts and all the other information from other atmosphere apps. It also provides a WebSocket connection for notifying about changes. That's why this whole thing works in real time. It's not just pull. Apps subscribe to your hosting, and they're notified when something changes so they can immediately ingest it and put it into their databases.
@TheoSteiner So for the traditional web, this would be, for example, the Twitter server. I want to post something to Twitter, and it goes to the Twitter server. But in ATProto, this is the PDS part, right? The Personal Data Server.
@TheoSteiner 従来のウェブだと、例えばTwitterのサーバーがそれにあたる感じだよね。例えばTwitterに投稿したいときは、そのままTwitterのサーバーに送られますよね。でもATProtoだと、ここがPDS、つまりPersonal Data Serverの部分なんですよね。
@danabra.mov Yeah. Yes.
@danabra.mov そうそう。はい。
@TheoSteiner And my data only lives on one server. So it's like a one-to-one mapping between my DID document and the PDS.
@danabra.mov Yeah. Your data always points to one. I mean, one server can serve many users' data, right? It scales, but your own stuff only lives in one place. At least the source of truth. Of course, apps consume it and have their own local caches or databases, but the source of truth is your PDS.
@TheoSteiner And I did a migration and everything.
@TheoSteiner で、マイグレーションも全部やりました。
11. Migrating hosting, and exploring a repo with PDSls
@danabra.mov Yeah.
@danabra.mov うん。
@TheoSteiner I think that's the interesting part you talked about earlier. Since your identity stays in one place, you can switch where your data lives and everything follows it around. So I had a Bluesky account. Initially, signing up left me with the default Bluesky PDS. Then I made a bunch of posts, a bunch of likes, followed, of course, Dan Abramov on Bluesky... And then at some point I wanted to move that to Europe. What happened behind the scenes when I did the migration?
@danabra.mov Yeah. The first thing that happened is that your entire repository—all of your JSON—was copied. If you're familiar with Git, it's kind of like pushing Git to a different remote. Your repository has been pushed to another place. You also had to migrate your media. Currently that's a little wonky. You have to use a separate tool to migrate all the image files. I think they're planning to improve this so the migration goes server-to-server. Currently it goes server-to-client-to-server, which is why it's a little strange. Ideally it would be much faster and a smoother process. Once that repository appears on the other server, the last thing that happens is you submit a change to the PLC directory. There's a request that says, "Update my passport—my DID document—to say this is now my hosting." This is now the HTTP endpoint where my content actually lives. That's the last step. That change is then broadcast from the PLC directory. I think the dominant model is that apps cache identities for something like five minutes or an hour. So the next time they resolve your DID they'll see, "Okay, the content is hosted somewhere else." But for most apps they don't even need to do anything because they've already cached all the data by its AT URI. The AT URIs didn't change. If they ever need to refetch something, they can simply go to your new server. So for apps it barely matters because most apps just cache the data.
@TheoSteiner So basically what I did was... I had a repo with all my data. It was on GitHub. Then I did a Git push to GitLab instead, deleted the old repo on GitHub, and updated my passport to say, "Hey, my data now lives over here."
@danabra.mov There's actually a great tool to inspect it. It's called **pds.ls**. If you literally type "PDS" as in Personal Data Server, and "ls" as in the Unix `ls` utility for listing files, it'll give you a page where you can input any ATProto handle. You can input your domain if you have an ATProto account associated with it, and it'll show you the whole repository. You'll see collections, which are kind of like folders. If you search for yourself, it'll resolve the DID, resolve the host, and you'll see it says **eurosky.social**. So it resolves where your hosting actually lives. Then these collections are kind of like folders. They're grouped by type. These are all different types of JSON. They're app-specific. The Bluesky app has its own types. npmx has its own types. Every app developer can come up with their own. Nobody is policing them. They're sorted by reverse domain, which naturally gives them unique prefixes. If you work on some app, you just put your app's domain there. It's kind of like a namespace. Almost like scoped packages on npm. And then if you go into any folder, you're going to see a list of JSON files, or records, in that folder that all have the same type. And it's just normal JSON. A like would be a JSON object that says which post was liked, for example. These are going to be links. So if you click into a like, or any specific piece of JSON, there's going to be a specific... For example, this is a like for a certain post. And if you follow the link to the post, you might end up on a different person's repository with the actual post.
@danabra.mov 実はそれを調べるのに便利なツールがあるんだ。pds.lsっていうツールだよ。「PDS」はPersonal Data Serverの略で、「ls」はUnixのファイル一覧コマンドのls、そのまま入力すると、ATProtoのハンドルを入力できるページが出てくる。ドメインにATProtoアカウントを紐付けていれば、そのドメインを入力するだけでリポジトリ全体が見られるよ。コレクションが表示されるんだけど、これはフォルダみたいなものだね。自分で検索してみると、DIDが解決されて、ホストも解決されて、eurosky.socialって表示されるよ。つまり、実際にどこでホスティングされているかが分かるんだ。で、これらのコレクションはフォルダみたいなものなんだ。タイプごとにグループ化されてるよ。これらは全部、異なる種類のJSONです。アプリごとに固有なんです。Blueskyアプリには独自の型があります。npmxにも独自の型があります。どのアプリ開発者も自分で型を作れます。誰かが管理しているわけじゃありません。逆ドメインで並べられているので、自然とユニークなプレフィックスになります。何かアプリを作るなら、そのアプリのドメインをそこに入れるだけです。いわば名前空間みたいなものですね。npmのスコープ付きパッケージみたいな感じです。それで、どのフォルダに入っても、そのフォルダ内に同じ型のJSONファイルやレコードのリストが見つかります。ただの普通のJSONです。例えば「いいね」は、どの投稿にいいねしたかを示すJSONオブジェクトです。これらはリンクになります。なので、いいねや特定のJSONをクリックすると、そこに特定の…例えば、これは特定の投稿へのいいねです。そのリンクをたどると、実際の投稿がある別の人のリポジトリに行くかもしれません。
@TheoSteiner Okay, so for the people listening to this podcast, I just navigated to **pds.ls**, and then I put my alias—so my domain, **yoshana.de**—on there and sort of did the DID lookup. Is that what you call it?
@TheoSteiner The PLC lookup. And it found my PDS, which in this case is **eurosky.social**. Below that, we had a bunch of records—no, wait, a bunch of collections. The collections are all of a certain type, I'd say. And the type is a reverse-domain identifier. So for Bluesky likes it is `app.bsky.feed.like`. If I click on that, I get all the likes I ever did on Bluesky. And if I click even deeper into that, I get the actual JSON. Here it says it has a timestamp, or `createdAt`, and then it has a subject, which I assume is the target of my like.
@TheoSteiner So URI is the thing we talked about earlier. This will resolve to somebody else's DID document, and from there we can get the PDS, and from there we can get the collection, and from there we can get the entry—the post in the collection. But what is the CID here?
@danabra.mov Yeah. So there are two types of links in ATProto. One is a weak link, or I think it's called the ref, which is just a normal AT URI. And then another one is called a strong link, which is URI and CID. CID is just a hash of the linked record. The idea is that for some links, you maybe want apps to be able to tell: is the thing that it relates to the same as it was? For example, for the like, if the post was edited, does that still count? I think the Bluesky app actually doesn't... I don't know. I don't think it reflects edits. I'm not sure. Maybe it doesn't. So this is an application choice to disregard edits, or it's a product decision. But if they ever allowed edits, this would allow you to say, "Was your like on the version that's there now, or has it changed since the like?"
@TheoSteiner Okay. So if, for example, I liked a post that said tabs are better than spaces, and then at some point the poster—the author of the post—went back and said spaces are better than tabs, I can point at this record and say no, because the hash changed. Back when I liked it, it was a different post.
@TheoSteiner And what is the `$type` field here in the JSON?
@TheoSteiner それで、このJSONの`$type`フィールドは何ですか？
@danabra.mov Yeah. So this is the name of the collection it's in. It's a convention that it's always going to be there. So it lets you tell just from the record what kind of record it is without even knowing the URI. It's just the folder name, basically.
@danabra.mov Yeah. So this is an interesting question. There are these things called lexicons, which are kind of like type definitions. I really think of them as a lexicon. For each of these types—Bluesky like, Bluesky post, Tangled repo, npmx follow, whatever—each of these things has a schema associated with it, written as JSON. The funny thing is, you don't have to publish them. They're useful for apps because there's tooling that generates actual parsing code and validation, or Zod validation, so that you can safely read it into a typed form in any language. There's JavaScript, Swift, Go, and so on. There are codegen libraries that do this from lexicons. But you can also publish lexicons, and the funny thing is the way you publish them is they're also records.
@danabra.mov So you can publish a record with the type definition. And actually, in **pds.ls**, if the type is published, you can click on it as a link and it will show the record. So on the screen you were showing, where you had the like record and it says `$type: app.bsky.feed.like`, if you press on it, there's a lexicon resolution mechanism which says how to find the record that has the schema for this type. And **pds.ls** actually displays it nicely, almost like documentation.
@TheoSteiner That's pretty cool. So this is how they allow storing arbitrary data, because there's a way to define how data is structured, and every app gets to define this themselves.
@danabra.mov Yes. Although it's even more interesting. I think that is the starting point: as a developer, you choose your own formats. But you can also choose to reuse existing ones. You can decide that you're making, maybe, a plugin for Bluesky and then use Bluesky follows as a format for your thing. Or maybe you're making some kind of annotation, like a Community Notes feature or something like this. In that case, maybe it would make sense for Bluesky to do it. But maybe you're making some kind of annotation thing, and then you could allow the target of your annotation to be Bluesky posts, Tangled repos, or anything. You can link to anything. So in that sense, you can use other apps' formats. You can link to them, or you can literally use them as your own format. It's the same as with file formats, right? Many apps use the same format. There's actually been precedent. There are three blogging platforms on ATProto. One is called Leaflet, another Offprint, and another is called pckt.blog. At some point they worked together on a single lexicon because they wanted it to be more interoperable, so that subscriptions from one show up in the other, and so on. They agreed on the format for a publication or a follow. This is called **standard.site**. It's become a bigger thing than this. It's kind of like RSS on ATProto in a sense. The funny thing is now you can also adopt standard.site for your own blog that's not hosted on any of these platforms, and create records describing your post, your publication, and your brand colors in the publication. And links to any standard.site posts in the Bluesky app will be shown with a special treatment, with a box that has the branding color of your site. So this is an example where three different blogging platforms decided to share a lexicon, and then the Bluesky app decided to read that lexicon and use the information from it. In that sense, it's a much more dynamic ecosystem where some things get standardized, in a sense, but completely community-driven.
@TheoSteiner This is kind of wild. This is like I built my own app, and to kickstart it, I let my users post from my app to Instagram. I let them post pictures to Instagram, and I can also display every picture they posted to Instagram. Yeah, this is the moment where it kind of clicks.
@danabra.mov Yeah. Well, the other thing is you can index things of different types, right? Of course, indexing all Bluesky posts is crazy in terms of volume, so that's expensive. But you could maybe index something that relates to the user. And then you can do these kinds of mashups. There's another funny example. There's this lexicon—do you know Last.fm, the music tracking thing?
@danabra.mov Yeah. So there's a lexicon called **Teal FM play**. Teal FM was supposed to be something where you track your music usage, but I don't think they've launched yet. It's been in development for a long time, but they're doing it on weekends. So they still haven't launched the actual website or the app. But it doesn't matter because the lexicon exists. They already made tooling to submit what you're listening to, even though there is no app. You're just writing records of this lexicon. And so people have already made third-party apps that display your listening stats on Teal FM, even though Teal FM does not exist. It's kind of funny. You can show aggregations across it, like top listened artists on Teal FM. But again, there's no Teal FM. The way I think about it is: if you had the world's JSON, why not run joins over products? You can do things like top tracks that your follows on Bluesky have been listening to. Or you can ask: what are the trending repositories from your friends? You can combine data from different apps and cross-join it. That feels like the web to me. It's taking the web idea and turning it into a web database. I don't know. It's breaking the boundaries, or blurring the boundaries, between apps, and making it more like there's the web of data, and the apps are just lenses through which we look at the data. And these lenses can be... anyone can create a new one.
@danabra.mov だよね。それで「Teal FM play」っていうLexiconがあるんだ。Teal FMは音楽の再生履歴を記録するサービスになる予定だったんだけど、まだリリースされてないみたい。ずっと開発中で、週末だけ作業してるらしい。だから、実際のWebサイトもアプリもまだ公開されてないんだよね。でも、Lexicon自体はもう存在してるから問題ないんだ。アプリがなくても、今聴いてる曲を投稿できるツールはもう作られてる。つまり、このLexiconのrecordを書き込んでるだけなんだよ。それで、Teal FM自体は存在しないのに、再生履歴を表示するサードパーティ製アプリもすでに作られてる。ちょっと面白いよね。例えば、Teal FM上でよく聴かれてるアーティストの集計とかも表示できる。でも、やっぱりTeal FM自体は存在しないんだよ。僕のイメージだと、もし世界中のJSONがあったら、プロダクトをまたいでjoinできるよねって感じ。例えば、Blueskyで自分がフォローしてる人たちが最近聴いてる曲のランキングとかもできる。あるいは、友達の間で今話題のリポジトリは何か、とかも調べられる。いろんなアプリのデータを組み合わせて、クロスジョインもできるんだ。それって、まさにWebっぽいよね。Webの発想をそのままWebデータベースにした感じ。わからないですね。アプリ同士の境界を壊したり曖昧にしたりして、データのウェブがあって、アプリはそのデータを見るレンズみたいな存在になる、そんな感じですね。そしてこのレンズは誰でも新しく作れるんです。
14. Relays and the authenticated firehose
@TheoSteiner And posting records across products sounds kind of crazy at first, but since it's the AT Protocol—Authenticated Transfer—that means you have authenticated everything that is posted to your repo, and that's why it's safe to post across apps, right?
@TheoSteiner プロダクトをまたいでレコードを投稿するって最初はちょっとクレイジーに聞こえるけど、AT Protocol、つまりAuthenticated Transfer Protocolだから、自分のリポに投稿されたものは全部認証されてる。だからアプリ間で投稿しても安全ってことですよね？
@danabra.mov Yeah. So the way this works is apps can be sure that the content is really yours because they can always resolve it through the AT Protocol. They go to your hosting, and your hosting returns them the thing. It is signed with the key that they can verify is the key in your DID document. And then the other clever thing they do is that you need to somehow scale up the network. You can imagine for each user there is hosting. Of course, a lot of users are going to be on the same hosting. Hosting can range from hosting 100,000 users to maybe a solo user hosting. Both of these are valid. But that's where the data lives. Then you somehow need it to flow into all the apps, right? And it's inconvenient for apps to... You don't want them to smash all the hosting all the time to figure out what happened. So apps maintain caches and subscribe through WebSocket. But then there's a question of: how do you scale it further? You want some shared infrastructure. You want shared caches. You want shared retransmission. Ideally, there should be a way to just subscribe over WebSocket to one event bus that has information about everything that happens on the network, or a subset that's relevant to your app—maybe your app's collection types and so on. The clever thing they did is that you can build retransmitters like this. They're called relays. Bluesky runs one. There are a bunch of community-run relays that let you subscribe over WebSocket to everything that happens on the network. The stream that is passed is a stream of commits. It's a stream of events about commits, identity changes, and so on. And the stream is transmitted with proofs. The stream contains signed commits. So each commit is signed. You actually don't need to trust the retransmitter. That's another big idea. Because cryptography is woven into it, you don't need to trust it. You can build community retransmitters and these nodes, and not have to worry that they're modifying the content as it flows through them. So that's the authenticated part in it.
@TheoSteiner So if I try to recap this for my understanding: You have a very weird distribution of Personal Data Servers. There are some with millions of users—probably the Bluesky ones—and then there are people who host it in their basement. And to not have every app know every PDS, go there every time, and pull them all the time, you have those relay services. I imagine it's a super beefy server somewhere that somehow knows all PDSs and subscribes to all of them. Then it's just like a huge Apache Kafka, and they pipe the events into the web.
@TheoSteiner じゃあ、僕なりにまとめてみるね。Personal Data Serverの分布がかなり独特なんだよね。何百万人もユーザーがいるPDSもあれば、家の地下室でホストしてる人もいる。たぶんBlueskyのPDSが前者だよね。そして、すべてのアプリが全PDSを毎回知ってアクセスし続けるのは現実的じゃないから、Relayサービスがあるってことだよね。たぶん、どこかに超ハイスペックなサーバーがあって、全PDSを把握して全部に購読してるイメージかな。それって、巨大なApache Kafkaみたいなもので、イベントをウェブに流してる感じだね。
@danabra.mov Yeah. I don't think it needs to be super beefy. I'm not sure, but in terms of cost, I think it's something around 30 bucks a month to run it at the current volume. But yeah, they need to know the PDSs on the network, which is a bit wonky because how do you know? There are a bunch of ways. One is that you can enumerate them. I think there is an endpoint on existing relays to enumerate the ones that they know. That's how you can bootstrap. A PDS can also request to be crawled. So if you put a new PDS online, you can add some known relays to ping them. And then it can also discover it through links. If it sees a link to a PDS it hasn't seen before, and that resolves to a PDS it hasn't seen before, it can decide to crawl it. So it's kind of up to the relay how to do it. And if it wants to just provide an archive or only retransmit and have no archive—because keeping an archive is kind of expensive...
@TheoSteiner So if I want to build on ATProto, I have this shared data layer, and I have apps that are already quite successful that populate that data layer with users. Those users use those apps and therefore populate it with their own records. And if I want to build on that, I would probably get one of those relays and start downloading data from there, putting it into my own database. Is that how it works? And then I run queries and my app logic on that?
@danabra.mov Yeah. So I think the exact way you do it depends on what data your app is going to care about. If it only cares about its own data—if you came up with new record types and that's the stuff that your app displays—then you don't need to backfill it because it doesn't exist. So if you just connect and start listening to a relay, and you filter it down to the stuff that your app cares about, then once records start getting created, you will have all of them. You don't actually need to get anything that existed before. That's the easy path. Then you just have a connection to this thing, and you keep track of where your users are. If you want to do something with existing data, then you have to backfill it. There are really two ways to go about it. The way I'm talking about is: if you want data to be in your own database, then of course you have to somehow get it there. Then the question is: is it from a big app or a small app? Because if it's something like Tangled, you can probably backfill all records from all Tangled users in a reasonable time because there's just not that much. Maybe 50,000 records. I don't know how many, but it's not that much. Or Teal FM. Okay, maybe a lot of records because people do scrobble. It's probably millions now. But essentially, you just need to find all the... I think maybe the relay has an API for this, but there's a way to find out which users or identities have a collection of a certain type. So you can figure out who the users of a particular app are and then just get their stuff. There are tools that do this. There's something called **TAP**, which is their tool for backfill. You tell it which collections to look for, and it will try to synchronize it all and then listen to new events. But there's also a different approach. Especially if you want Bluesky, you probably don't want to download terabytes of all Bluesky posts that ever existed. But maybe you want to build some app on top of Bluesky. Of course, you can use the Bluesky API. That's the traditional way, but I think it's a bit not ATProto-spirited, right? You rely on somebody else's app-specific thing. But there's a tool called **Constellation**. It's community-run. It's a server run by the community that indexes all the records and provides an API for backlinks. So it lets you say, given this post, give me all records that link to it. That means you can get all the likes of a post, all followers of a person, all quotes. It's kind of like going through the primary-foreign-key query. You ask: what are all the things that link to this piece of JSON? This actually lets you build a workable app without a backend, because you can hit the user's hosting if you need something user-specific, and then you can hit Constellation if you need to go one step removed and see all the things that link to this. This is less efficient, but you don't need a backend at all.
16. Collapsing the stack: OAuth, cost, and reliability
@TheoSteiner Yeah. So the more I hear about this, at first I thought AT Protocol was really complicated. But now I understand why it needs to be complicated, because it collapses the stack. My app needs less and less. It doesn't need a data layer necessarily, because every user has their own data layer, and I can use the existing relays and everything to aggregate that. Then it doesn't need an authentication layer, because the user has—is it called an atmosphere account?
@danabra.mov Yeah. So it's kind of funny because it uses OAuth, right? You log in through OAuth, but the OAuth provider is actually your PDS. So in a sense, it's almost like logging into a Google account, except you are Google. It's your hosting. This is why, if you change to **eurosky.social**, when you click "login through atmosphere" in other apps, it's going to show the eurosky.social login page now, because that is now your hosting.
@TheoSteiner That also brings its own complexity though, right? For example, let's just take an Instagram competitor. You want to build that app, and it's slow, and you want to debug or improve performance. But the problem could be outside of your product. It could be the user's PDS being really slow or laggy. It could be the relay struggling to aggregate the data. How do people cope with that?
@danabra.mov Yeah. I think with hosting itself, the idea is pretty resilient to that. If you want a good app, you just shouldn't really go to hosting much, because you aggregate things as they come in. So the hosting just emits events. You shouldn't blast the hosting with a thousand requests. You serve everything from your database or from the stuff that you've cached. So hosting being a bit wonky is actually fine, as long as it actually emits the events that it should generate. That isn't really a major problem. It's just a server that serves some JSON, so it shouldn't really get laggy. Of course, for users who are on Bluesky, if Bluesky is having scaling issues, then that does reflect in the app. Some events might struggle to come through. The relay might struggle. This is something you just kind of live with. Your app is more webby, so there are more things that could go wrong. The relay itself—again, if you rely on the one run by Bluesky, it's mostly working fine. I think there have been incidents, but it's mostly working fine. If you don't trust it, you can definitely run your own, and that is how you completely isolate yourself from problems upstream. There were also cases where other apps that run their own relays or used the community relay were up when the Bluesky app was not working, because there was some relay problem. I think with time... Right now it's the early adopter stage, where you run it because you believe there is a better way to do web products. I think of it as open source in the 90s. People would tell you you're crazy, or Microsoft says it's cancer, and then 10 years later Microsoft is actually running on open source. So I think right now we're at that stage where people tell you you're crazy. These are early adopter pains. But because it's a standard, everything that benefits some apps can benefit others because they speak the same language. There's shared infrastructure, shared libraries, shared tools, different kinds of things you can do with it. As that infrastructure grows and becomes more reusable, the network will become more resilient in general. There will be more relay options. It'll be easier to backfill. There will be archives for rehydrating everything if you lose some data. So I think we'll see more of that.
@TheoSteiner Yeah. I actually just thought of a really interesting problem concerning sustainability. You talked about Bluesky being both the app layer and the PDS layer. For Bluesky, I can clearly see a path to monetization. They have the app layer, and it would be quite easy for them to monetize that. Let's say display ads or premium accounts. But there are PDSs that don't have an app layer, and I'd say they bear most of the cost because they have the egress and hosting costs. Is there a way for apps to give back to the PDSs, or somehow shift the weight of that cost structure?
@danabra.mov Yeah. I'm not sure. I think running an app is many times more expensive than running hosting. I'm not 100% sure, but first, the app serves the users. If you have a million daily users, that's the load that goes into the app. The hosting serves the apps. So that's actually much more distributed already. Especially if there are different hosts. If you self-host, you really only serve your own data. Compared to something like Mastodon, where if your post goes viral, all the instances are going to hammer your instance. Here it doesn't really matter that much because the app shields you. The other thing is that I think hosting is still such a novel concept. Usually you have hosting if you have a website or a company. But as part of your internet experience, even if you log in with a Google account, that's kind of the relationship you have to Google. The idea is that we can lift this up and make it transferable. I think it's kind of new. There's a Dropbox with all your social data. That's kind of a new idea. I don't think it's even recognized by mainstream users at this moment. Maybe in five years, or maybe in ten years, it will be. But hosting can also offer services. I'd actually pay for safe backups. I'd really want my hosting provider to periodically back up all my data without me thinking about it. That could be one thing. Some people might want more storage, maybe for videos and things like that. But in general, I think it's actually the app that's much more expensive. Things like CDNs, video transcoding, serving lots of video—all of that is way more expensive than serving JSON. That's where I think a lot of the costs actually go.
@TheoSteiner So you're saying that in order to run a stable app, you want to shield the PDSs from your app layer anyway. And shielding the PDSs will take most of the load off the PDSs and onto your app layer. So it's sort of a self-restraining system.
@danabra.mov Yeah. Because you want it to be reliable. If a PDS is misbehaving, you don't want that to reflect in the app. Past a certain level of product complexity, you just have to have a database. Or you can hit community caches, but they're shielding you. If you hit Constellation, that also shields hosting because Constellation has a cache of everything that it has indexed.
@danabra.mov Yeah. There are a lot of rough edges in terms of developer experience, or what the right way to do something is. People are still experimenting. But I think the big thing the team is currently working on is permissioned data. Right now, ATProto is fully focused on public data, which I think was the right focus. If you start with private data, that's a much smaller scale. You don't know that you're going to scale up to public. Whereas if you scale up to public, you have to figure out aggregation and all these things efficiently. Then you can scale down to permissioned data. So they're working on extending the protocol—or maybe having a sibling protocol—for stuff that isn't meant for public retransmission. Things like groups. Not messaging, because for that you really want end-to-end encryption, and that's an entirely separate thing. You could still use part of the identity layer, but it's a separate problem. Whereas things that are gated by membership aren't completely private, because any app that has been granted access still needs to see them in order to display them. So they're working on this permissioned-data extension to the protocol. Right now, if you want something private or group-private, you can do it with your own database. Which is also okay. ATProto is pragmatic in the sense that you can adopt just the parts that make sense to you. You can use only the identity layer and keep everything else in your own database. Ideally you'd move towards "ATProto-ing" it a bit more. But then you run into the wall where group-level stuff, user preferences, and things like that have to live somewhere. There's a hacky thing where I think you can store some of it in Bluesky preferences. But anyway, there's no good API for private or protected data. That's what they're working on. I think that's the biggest gap. There's currently no protocol-native way to do that.
@TheoSteiner So this might be a sacrilege in a podcast with a React core member, but I'm a huge fan of Svelte. In the ATProto world, quite a few apps seem to be built with Svelte. There's someone—or maybe a group of people—trying to build a Discord clone based on ATProto. I think it's called Roomy Chat or something.
@TheoSteiner And they seem to be stuck on having private servers because they don't have permissioned data yet. Having permissioned data would unlock that for them because they could have data being created behind access controls.
@danabra.mov Yeah, probably. I think so. Things like chats are just a different modality because they're kind of ephemeral. Maybe for Discord you do want to keep it, but I don't know that you necessarily want every chat message to be a record. I don't know if that makes sense. But having permissioned data would definitely help. Svelte is really popular in that part of the community. I'm not sure why. Maybe it just fits the ethos. It's kind of webby. It has the connotation of being the lighter thing, the newer thing. And React is like Java to these people. Which is okay. I'll take it. Whatever works.
@danabra.mov I don't know. I would encourage more people to actually try Bluesky for real—for posting the stuff you would normally post on Twitter. One thing I would encourage is to actually... The default feed, I think, is called Discover, and I don't think it's very good. There is a very good custom feed called **For You**, literally like the "For You" feed on Twitter. If you go to the Feeds tab in the Bluesky app and search for it, you'll find it. I highly recommend it. It learns from your likes and from what people who liked the same things as you have also liked. I honestly feel it's about as good as the Twitter algorithm used to be. I always use that feed. The other funny thing is that this feed literally runs from somebody's gaming computer at home. Again, because what is a feed? It's just an HTTP service that returns an array of AT URIs for posts. Everything in the network is identified by an AT URI, so it doesn't matter where it's hosted. You can build these services because they're all talking about the same thing. This For You feed just returns an array of AT URIs, paginates through them, and then the Bluesky app talks to that feed, gets the URIs, and hydrates them from the database into actual posts. It's very cool that you can have userland feeds on Bluesky. I think this is the best one. Especially if you're trying to get people to move over and try Bluesky, just tell them to install the For You feed because it's simply better.
@TheoSteiner I'm also on For You. And it's super funny because sometimes the person who hosts the feed goes down for an hour, and then it comes back up and he posts something like, "Sorry, I had to upgrade my PC. I put more RAM into it."
@TheoSteiner Okay. I think that's it for today then.
@TheoSteiner なるほど。今日はこのへんで終わりにしましょうか。
@danabra.mov Yeah.
@danabra.mov そうですね。
@TheoSteiner Thank you so much for coming on and talking about ATProto. I mean, you obviously couldn't say it yourself, but probably the best way to learn about ATProto is your blog. You have a whole series of blog posts on ATProto and how it works, using different ways to break it down. I think the file system one is great. Then go on Bluesky and just start following a bunch of people. My personal recommendation would be the Starter Packs. They let you, with just one click, follow everyone in your area of interest.
@TheoSteiner I'm in the Svelte Starter Pack, which was really cool because it got me bootstrapped with a whole bunch of likes, which I was really happy about.
@TheoSteiner And actually, this show is also on Bluesky—or if it isn't yet, I'll make sure it will be after this post. You can always check us on the regular HTTP web. Our domain is **uit-inside.linecorp.com**. Make sure to follow Dan if you don't already. And thank you so much for listening. If you want to listen to other episodes, most of them are in Japanese, but I think there are three or four English episodes out there. So just look for them. Or maybe study Japanese. Or use a tool to translate them. Thank you so much for coming on. Bye-bye.
