Internal PuSH hub in Friendica to solve posting to StatusNet

For a long time now, there has been a problem in Friendica with posts not being pushed out to subscribers on StatusNet (or its successor GNU Social). The bug is documented on Friendica’s issue tracker and has been discussed several times.

The problem is that StatusNet expects a protocol called PubSubHubbub (or PuSH for short), which basically is a hub where a client can subscribe to a particular topic (a particular Friendica user in this case) and supply a callback URL. When the topic gets new posts (Friendica user posts new stuff) it notifies the subscriber to the designated callback URL and actually pushes the new data to subscribers, instead of them having to constantly check for new stuff.

Friendica has until now relied on Google’s public PuSH hub for this, but it has recently stopped working reliably with StatusNet. I’ve later learned that problem is most likely due to the fact that StatusNet doesn’t refresh its subscriptions, but many people also don’t like that Friendica uses an external service for this.

I also tried using another third party PuSH hub, but without success. So I thought that it would be best to implement it internal to Friendica which would give me more control, and it would not rely on a third party service.

So, I finally had some time to sit down and make a crude implementation. I’m really new to Friendica development, and PuSH is unfamiliar to me as well. My implementation is now live on my Friendica instance at social.saz.im and it seems to work, at least with a small number of subscribers. I’ve tried to make it conform with both the 0.3 and 0.4 versions of the PuSH spec, except I do not implement all options, and it doesn’t actually expire subscribers after the refresh time.

The actual pushing out of new posts is done in Friendica’s poller, which isn’t running in the main Friendica program, but as a cron script that is launched regularly. This also means that pushes will not be instantaneous.

I intend to start following people from StatusNet/GNU Social again from my Friendica instance, and hopefully some people we follow me back and get my posts pushed to them. One final problem remains, though: StatusNet sites that have been subscribed to me before will still have a record in their feedsub table pointing to the old Google PuSH hub, and it will continue to use this even if people subscribe to me again. Administrators of those sites would need to force StatusNet to refresh the hub to the new one by running a command similar to this:

php /path/to/gnu-social/plugins/OStatus/scripts/update-profile.php \
    https://social.saz.im/profile/sazius

(of course replacing with the correct path).

My internal PuSH implementation currently resides in the internal-push branch in my Friendica clone on github. I’d be happy if anyone more experienced with Friendica development could take a look at the code and say if it makes sense and is reliable enough. I’m not sure how well it will handle hundreds of subscriptions and subscribers that go offline and things like that. Right now it tries N number of times (on different poller runs) to notify a subscriber that is offline and then gives up.

Update

My PuSH fix was merged into Friendica master branch on Nov 26, 2013. If you are running a newer version, or revision 3.2.1746 or newer you should have my fix. It still isn’t automatic, though, you need to have this line in your .htconfig.php in your Friendica directory:

$a->config['system']['huburl'] = '[internal]';

Remove or comment out any previous lines setting the ['system']['huburl'].

Posted by Mats Sjöberg.