Fediverse

Integrating a News Publication Into the Fediverse

A thorough review of trials, tribulations, and tests to bridge the gap from publishing online to federated social networking.

Sean Tilley

Sean Tilley

— 9 min read
A man wrapped in newspaper from head to toe, reading a newspaper in front of more newspapers.
Photo by Denise Jans on Unsplash

Cards on the table: I've been experimenting with integrating my news publication project with the Fediverse for nearly half a decade. There have been a variety of experiments, mistakes, and teachable moments that have gotten me to where I am today. I want to talk about some of the things I've tried, along with why these things were important to me.

Why do this?

We Distribute is kind of a weird experiment built around a weird idea: what is we built a news publication that reported on the Fediverse as its main focus, but was also fundamentally a part of the Fediverse itself.

I thought this was worthwhile for the following reasons:

  • Self-Demonstration: the medium and the message are entangled in such a way so as to demonstrate the concept. In addition to RSS, you can subscribe to the publication directly through ActivityPub.
  • Timeline Integration: following a publication could work similarly to how people follow a Facebook page, but with no middleman. Articles can show up in people's timelines in a relatively native way, and their replies show up on the actual article page in the comments section.
  • Content Production: no one else was really doing anything like this at the time. To me, it felt like no one else was doing this, and I thought it was important to try to reach people where they were at. Keep in mind, this effort started back in 2015, when the Fediverse had virtually no publishers on it.

This experiment emerged in three distinct phases:

  1. Build on top of an existing federated platform and try to do publishing with it.
  2. Build on top of a Content Management System, and try to leverage some kind of integration to do the work for me.
  3. Take the proof of concept from #2, but really ensure that the base is solid. Then, explore deep integration.

Case Studies

Here's a couple of attempts at this thing over the years. As I continued to build and explore, a lot of ideas ended up solidifying into a unified, coherent idea. However, it took a lot of experimenting to get to that point!

First Attempt: Hubzilla

The first effort of this kind involved a platform called Hubzilla. As a platform, Hubzilla is wild: what if you combined a groupware system, a content management system, a cloud storage provider, and a social network all into one thing?

Hubzilla, with some of my own custom Facebook-like theme on top.

Hubzilla had a lot of forward-thinking ideas, but the platform is notorious for its complexity and lack of a unified design. It's a system where every feature has its own set of special settings menu tucked away somewhere else. At the time, the platform's creator was experimenting with implementing support for publishing articles, and an early version of We Distribute tried to use this. Eventually, I chose to move on from Hubzilla. It felt clunky and slow, and wasn't exactly optimized for reading or writing articles.

Second Attempt: WordPress and Pterotype

Our second attempt emerged out of a desire to move away from Medium. I had used it for a while because I was struck by its design philosophy and ease of use. But, I wanted to try experimenting some more, and had just discovered a new integration effort called Pterotype.

Initially, Pterotype worked great! The plugin basically generates a Note containing a description and link to an article, which is not unlike how a brand page looks on Facebook or Twitter. The blog just operates as an Actor that people subscribe to. Setup and configuration were minimal, and integration felt pretty easy.

Unfortunately, we ran into problems: there were some bugs, and the key guy working on this thing noped out after a couple of releases. Some volunteer contributors were effectively keeping this afloat for a while, but the project lacked any kind of contingency, and we had no migration plan.

Third Attempt: A Sea Change

Some time later, I discovered a new effort to bring ActivityPub to WordPress by Matthias Pfefferle. There were some feature differences, a lot more configuration options, and the plugin wasn't a 1:1 match with Pterotype. It would require some configuration on our part. Still, we decided to rip the band-aid off.

How did we manage to migrate from one plugin to another and retain our followers? Great question! We didn't. We effectively started over from scratch (basically had to republish all of our articles), and gave plenty of feedback about what we hoped to accomplish.

Matthias would go on to get hired by Automattic to work on the WordPress integration full-time, and his work would lead to all WordPress.com blogs having the ability to use ActivityPub federation. As the project matured, and Matthias began working with other experts in the WordPress community, a few interesting plugins developed:

  • Friends - a cute little IndieWeb-style social networking tool that can also leverage the ActivityPub plugin to do things like submit follow requests and replies to your friends.
  • Enable Mastodon Apps - Gives WordPress a Mastodon-compatible API that Mastodon apps can use.
  • Event Bridge for ActivityPub - Maps existing Event plugins for WordPress to the ActivityPub Events schema for compatibility with Mobilizon, Gancio, and any other ActivityPub platform that supports the Event activity.

The main plugin itself has become quite sophisticated, to the point that we had to write a guide explaining to other people how to get basic usage out of it. Over the years, we've submitted bug reports and feature requests, and the team has helped us to figure out how to accomplish the things we've wanted:

The biggest challenge for us on this project has involved debugging. Our system relies on all these individual plugins and libraries, and it sometimes results in behavior that's hard to pin down. If the timeline stops working in an app, or a notification breaks, who do we file a bug report to? However, the devs behind these different projects are very good at sharing context with one another, and have led to lots of fixes.

Vision for the Future

So far, you've seen me name and link to a bunch of things, and are left wondering: what does all this actually do? Fair enough, I'll wind up the pitch.

Right now, the integration work has allowed for some really interesting stuff: we're finding new ways to blur the lines between publishing, social networking, and federation. Projects such as Ghost (which I now use for my personal blog) are taking a similar approach with their own code. They're just doing it in a more holistic fashion.

Publishing

One of the first major headaches we had in trying to build out our integration involved activity types: we wanted to offer native Article activities to any publishing platform capable of reading them. To us, it made sense that someone would eventually develop a federated social reader application, so treating Article and Note as potentially two separate things made sense.

For a while, our statuses looked like this: just a bunch of image attachments and a link. This happened because Mastodon doesn't support Article, and Mastodon didn't have very good fallbacks.

Unfortunately, this kind of thing just didn't exist for the longest time. So, we were stuck between capitulating to Mastodon, and just making everything a Note activity (which Mastodon uses for Statuses) or remain hopelessly incompatible. Then, Ghost happened.

Ghost is doing some absolutely wild stuff with federated publishing. They've been very public about their development work, showcasing a lot of stuff early and out in the open. Thanks to their adherence to open standards, our little WordPress site works almost perfectly with Ghost, except for the fact that our article thumbnails don't show up in the inbox. Still, that's a pretty minor concern, given how good everything else looks.

There have also been some advancements with the ActivityPub plugin suite that we're using, where different kinds of content can map natively to the network. For example, we use a podcasting plugin, and those show up on our timeline in an embedded player.

As an additional bonus, fallback handling for unsupported Activity types has gotten a lot better, too. On the Mastodon side, our timeline mostly looks like native statuses with big preview cards, instead of what it used to look like. Other platforms can go one way or the other, usually defaulting into a long-form article format within the timeline, but this feels like the least bad alternative option.

Social Networking

So, I want to speak to two plugins that we've started using heavily in day-to-day experiments, and they're both by Alex Kirk: Friends, and Enable Mastodon Apps. These two things hook together to make something truly special:

That's right. Native social interactions in WordPress. We can follow people, boost statuses, and reply to more than just whatever responses trickle in to our own publication. There's still a bit of debugging to work through...for example, we don't currently have working notifications. But, Alex has been amazing to work with, and we're happy to give good feedback and data to him as we face specific challenges.

Native microblogging, as seen on Mastodon and other places.

What's especially crazy about all this is how we intend to incorporate our timeline directly into the site. It's an ongoing design problem, and we're taking a lot of inspiration from The Verge's StoryStream layout. Here's a couple of mockups that we hope to eventually bring into a new WordPress theme.

Mockup by Scott Kellum, who we commissioned for his design work and insights. He also did the redesign of our logo and color scheme.

We're still playing with ideas here. The biggest challenge is that our site's theme desperately needs to be rewritten from the ground up, and finding a balance between "local content" and "federated stuff" can be really, really hard.

Another, somewhat less cool-looking concept by myself. The social icons actually feel like too much here.

As an aside: because we can now start using most Mastodon-compatible apps, it's now possible for us to make use of Mixpost, which is like an open source Buffer alternative. One thing we've kind of struggled with has involved making engaging community posts through our social media account, because most of our focus is on writing and publishing articles.

Mixpost is pretty great, by the way.

This isn't going to magically change things, but we now have an integrated way to schedule statuses throughout the weeks and months, well ahead of time. We're going to continue to experiment with this.

Federation

The final puzzle piece involves finding new ways to natively conform to existing platforms. We're actively testing the ActivityPub Event bridge, but some of the existing platforms, such as Mobilizon, have issues with compatibility. However, we have been able to make events work with the Friendica family tree of platforms.

Federated events as seen by Friendica.

It's an ongoing effort. In the long term, we might evaluate writing our own platform from scratch, maybe leveraging something like Fedify for the basic building blocks. But for now, we want to try to test this integration work as thoroughly as possible, and turn our lessons into guides that we can give to other publishers on WordPress.

In Conclusion

I'll try to keep it short and sweet. 😄

  • Sometimes, you have to be one of the first to really try something, and run with an idea.
  • Years of experimenting has us almost to a point where we can blur the lines between publishing, social networking, and decentralized communications
  • It's probably better to make a purpose-built platform for what you're trying to do, rather than try to bolt publishing onto a federated system or federation onto a publishing system. That said - if you have to, do the second thing.
  • ActivityPub integration for both WordPress and Ghost are amazing efforts, and they're largely compatible.
  • Debugging protocol integration stuff is a lot harder when your app is actually using a handful of plugins, instead of just one library.
  • Some of the stuff we're doing is probably crazy and doesn't scale to tens of thousands of subscriptions, but it might help bring traffic into the main site in the short term.
  • We've managed to build something pretty special, and cultivate an even bigger picture of what we want in the near future.

Read more