spacecowboy

@spacecowboy17.bsky.social

Interests in ML and social aspects of tech. Building For You feed: https://foryou.club https://bsky.app/profile/spacecowboy17.bsky.social/feed/for-you Hobby project: linklonk.com

Last week I ran an A/B test in For You where the treatment arm had a higher penalty for popular items and items with more paths leading to them were boosted more. Statistically significant results: 0.6% more users liked at least something on a given day 4.7% more likes per user

A table of experiment results:
Metric	Control	Treatment	Difference	% Change
Users	58,367	57,747	-620	-1.1%
Requests	5,596,792	5,657,823	+61,031	+1.1%
Requests per user	95.89	97.98	+2.09	+2.2%
Total likes	3,304,534	3,427,841	+123,307	+3.7%
Liked % (per request)	21.54%	21.65%	+0.11%	+0.5%
Likes per request	0.5904	0.6059	+0.0155	+2.6%
Total show more	13,386	12,199	-1,187	-8.9%
Total show less	37,337	36,700	-637	-1.7%
User-days	272,775	270,174	-2,601	-1.0%
User-day liked %	66.45%
[65.80%, 66.54%]	66.86%
[66.34%, 66.94%]	+0.41%	+0.6%
[+0.0%, +1.4%]
User-day show more %	0.75%
[0.68%, 0.79%]	0.80%
[0.72%, 0.85%]	+0.05%	+6.7%
[-2.8%, +17.4%]
User-day show less %	2.85%
[2.68%, 2.91%]	2.87%
[2.71%, 2.91%]	+0.02%	+0.7%
[-5.1%, +6.0%]
Requests per user-day	20.52
[19.7436, 20.4654]	20.94
[20.1519, 20.8653]	+0.42	+2.1%
[-0.6%, +4.7%]
Likes per user-day	12.11
[11.5858, 12.1550]	12.69
[12.1086, 12.7420]	+0.57	+4.7%
[+1.0%, +8.3%]
Show more per user-day	0.0491
[0.0336, 0.0625]	0.0452
[0.0357, 0.0527]	-0.0039	-8.0%
[-39.6%, +23.5%]
Show less per user-day	0.1369
[0.1166, 0.1516]	0.1358
[0.1174, 0.1487]	-0.0010	-0.8%
[-19.2%, +17.6%]

I'm not nitpicking on @gruber.foo but clearly atproto needs more evangelizing, education, marketing if tech people like Gruber don't get things like this right: - uses "username@bsky.app" instead of "username.bsky.social" - refers to the domain name as the identity instead of just being a handle...

AT Protocol Developers@atproto.com · 2w ago

How is the emerging post-platforms internet like the early days of blogging? @gruber.foo sat down with @jimray.bsky.team to go long on this question, with a few other digressions along the way. atproto.com/off-protocol...

Hmm, spelling it out actually makes it easy to test out the opposite - what if Bluesky was 2x smaller, how much worse would For You become? To do that I just need to make For You only pay attention to the posts authored by users and liked by users with even hash values of the DID. Thanos style.

Reddit thread:
PosterA: I am more interested in an app that shows posts matching my interests, similar to the Twitter algorithm. Like Bluesky’s Discover or For You, but significantly better.
PosterB: What issues do you have with For You. I understand the Discover feed's issues but spacecowboy's For You feed seems to be pretty good at showing me my interests.
Me: 
I'm curious as well! One thing that is hard to judge is how much better For You would be simply if there was 10x more users on Bluesky - 10x more content that matches your interests and 10x more people who share your interests. That is - is it the algorithm issue or amount of content issue?

A common misconception about Bluesky is that it "has no algorithm". In fact it has many algorithms - custom feeds. You can choose which one to use. What does it look like? You can replace the built-in fyp called Discover with a custom For You that runs on my gaming pc in 23 seconds:

I finally looked into why the For You sqlite db file has been growing to be quite large - 589GB. To prevent the unlimited growth I regularly delete likes older than 90 days. The size of the "ratings" table should be stable (there are ~20M likeds per day). Turns out another source of growth is...

Ahh, For You is returning errors because Bluesky added another ip address from which to make requests (38.142.8.130) and I need to give it a higher QPS limit in nginx. I wonder why nobody mentioned this yet...

I showed this survey post to For You users over the last 9 days. 1099 people submitted their response. Thanks everyone who participated and replied to the post with more details! I will spend some time analyzing the numbers and will publish the results on blog.foryou.club

Graph of responses per day starting from 350 down to 50. Total 1099 responses.

The power law shape of the graph is very cool to see even for such small numbers.
For You Feed@foryou.club · 2mo ago

Hi For You users! Please help me figure out what a 🔋Turbo version of For You would be worth to you. It works twice as hard to find the best posts. Answer one question in cups of coffee. Zero cups is a real answer. You get a month of 🔋Turbo either way: foryou.club/turbo

It looks like we are back for real this time. Curious to read the postmortem. Here is the shape of the outage from the perspective of the For You feed. There were a couple of recovery spikes. Jetstream was also down and recovered a bit after the app started working again.

For You users / minute graph with the values dropping from 700 to 400 during the outage.

I'm thinking of how For You could help the cold-start problem for new posters here. If nobody has liked your posts yet then you would not show up on For You. I think there could be a mechanism that helps people get discovered by the right audience and that is resilient to gaming/abuse. Ideas?

Custom feeds can mark some posts as pinned. For You uses it for system messages like "Welcome to the feed" and such. But AppView ignores that flag when it hydrates the feed and the client app does not know which posts should be displayed as pinned. This should fix it: github.com/bluesky-soci...

Propagate skeletonReasonPin from custom feed generators by spacecowboy37 · Pull Request #5163 · bluesky-social/atproto

When a feed generator returns app.bsky.feed.defs#skeletonReasonPin as the reason for a skeleton item, the AppView silently drops it. The skeletonFromFeedGen function in getFeed.ts only maps skeleto...

github.com