Wrapup: Make Sone scale - fast, anonymous, decentral microblogging over freenet

Sone¹ allows fast, identi.ca-style microblogging in Freenet. This is my wrapup on a discussion on the steps to take until Sone can become an integral part of Freenet.

Current state

• Is close to realtime.

• Downloads all IDs and all their posts and replies → polling which won’t scale; short term local breakage.

• Uploads all posts on every update → Can displace lots of content. Effective Size: X*M, X = revisions which did not drop out, M = total number of your messages. Long term self-DDoS of freenet.

Future

• Is close to realtime for those you follow and your usual discussion group.

• Uploads only recent posts directly and bundles older posts → much reduced storage need: Effective size: B * Z + Y*M; B = posts per bundle, Z = number of bundles which did not drop out, Y = numbers of not yet bundled messages; Z << Y, B << X, Y << X.

• Downloads only the ones you follow + ones you get told about. Telling others means that you need to include info about people you follow, because you only get information from them.

Telling others about replies, options

• Include all replies to anyone which I see in my own Sone → size rises massively, since you include all replies of all people you follow in your own Sone.

• Include all IDs from which you saw replies along with the people they replied to → needs to poll more IDs. Optionally forward that info for several hops → for efficient routing it needs knowledge about the full follower topology, which is a privacy risk.

• Discovering replies from people you don’t know yet: Add a WoT info: replies. Updated only when you reply to someone you did not reply to before. Poll people’s reply lists based on their WoT rating. Keep a list of people who answered one of your posts and poll these more often. Maybe poll people instantly who solve one of your captchas (your general captcha queue) → new users can enter quickly. When you solve captchas in WoT, preferably solve those from people you follow.
  → four ways to discover a reply:

  1. poll those you follow,
  2. poll the people who posted the latest replies to you (your usual discussion-group),
  3. poll those who solve one of your captchas (get new people in as fast as possible) and
  4. poll the replies-info from everyone with the polling frequency based on their WoT rating.