cross-posted from: https://reddthat.com/post/52386265

Right now, big communities dominate the feed. I’m wondering what sort algorithm could level the field so niche or hobbyist communities have a fair chance to get seen.

There’s a good related post: Niche Communities won’t be able to reach their true potential until Lemmy adds a sort that takes engagement into account. It puts it well:

“If Lemmy is to truly start having active hobbyist communities instead of being 95% lefty US politics, Shitposts, and some tech stuff, it needs a sort that takes into account the user’s engagement.”

What do you think should be the default sort for a more balanced Lemmy?

        • Die4Ever@retrolemmy.comEnglish
          12·
          9 months ago

          Maybe the size of the community should be determined based on posts per month instead of the number of active users

      • MHLoppy@fedia.io
        81·
        9 months ago

        Imo that’s as much a problem of the sorting algorithm as it is with a single community blasting out too many posts at once without any consideration for how current frontends are unable to usefully integrate that into people’s feeds. A couple of years ago nanoUFO was being a bit (subjectively) overenthusiastic about posting - I counted and [email protected] had 40 posts at once from them and it made the first couple pages of my feed basically unusable for a while.

        Scaled is also probably better suited for your subscribed feed rather than /All.

    • Avid Amoeba@lemmy.caEnglish
      11·
      9 months ago

      Scaled with Subscribed, then liberally subscribe to any and all communities I think I might be interested in. Check the Scaled + All firehose once in a while and subscribe to new communities if anything new shows up.

    • schnurrito@discuss.tchncs.deEnglish
      21·
      9 months ago

      That would cause me to miss many interesting threads that were created at a time when I happened not to be looking at Lemmy.

      “New comments” it is for me, that causes threads to get bumped to the top as long as other people still find them interesting.

  • PumpkinDrama@reddthat.comBanned from communityOPEnglish
    12·
    9 months ago
    1. Compute raw post score (upvotes minus downvotes).
    2. Normalize score by community size (e.g., divide by square root of active users).
    3. Calculate z-score relative to community mean and standard deviation.
    4. Apply time decay to prioritize recent posts.
    5. Sort posts by adjusted z-score.
    • Outcome: Posts that significantly outperform their community norm appear prominently, giving small and large communities equal visibility potential.
    • Enhancements: Minimum engagement thresholds, Bayesian shrinkage for small communities.
      • PumpkinDrama@reddthat.comBanned from communityOPEnglish
        12·
        9 months ago

        Nah it only rewards communities with few members, that means bot communities with lot of posts and almost no active users are always going to be top. That’s not the outliers sorting that I want.

    • Chozo@fedia.io
      151·
      9 months ago

      It’s not necessarily about competition, it’s about visibility. If I create something and I want to share it with people, that means I want people to see it. It doesn’t necessarily mean “I want people to see this more than other posts”, just “I’d rather not be posting into the void”.

      For instance, I make YouTube Shorts for a game I play. I don’t post them on Lemmy anymore, because the Lemmy community for the game only has 60 subscribers, most of whom aren’t even active accounts anymore. The highest-upvoted thread in the community has 47 votes, the second-highest only has 9. This translates to effectively nobody on Lemmy seeing the videos I made, because this small, slow-paced community’s posts get drowned out by everything else.

      • Skavau@piefed.socialEnglish
        3·
        9 months ago

        What’s the game? I feel like a broader community for “YouTube Shorts for a games people play” would have more successful than a community tailored to that specific game.

    • PumpkinDrama@reddthat.comBanned from communityOPEnglish
      47·
      9 months ago

      I think it is, and niche has lost to shitposting. Also, don’t tell me what to do.

  • Jayjader@jlai.luEnglish
    11·
    9 months ago

    “Top over the last 6 hours” can be a decent middle ground between “new”, “scaled”, and “active”.

  • OpenStars@piefed.socialEnglish
    102·
    9 months ago

    I tried and tried and tried and tried and tried and tried and tried and tried with Lemmy. The best I found that I could hope for was sorting by New, but mostly I just gave up hope for it.

    Until I moved to PieFed, and now the issue has multiple solutions. For one, using the Topic/Feeds (which are user-customizeable and shareable) you really can have your cake and eat it too, e.g. you can unsubscribe from all politics communities so that those do not show up on your main homepage, but an entire new feed completely dedicated to News & Politics is just a click away. Or Memes. Or Hobbies. Or Movies & TV, or any of a thousand other things - again, you can build your own, or subscribe to one that someone else has made.

    And for another, for sufficiently low-traffic communities you can click the bell icon (which you can do to pretty much anything - users, posts, comments, communities, etc. - plus you can even UNCLICK that to silence notifications from your own content!!), so that you get a notification for each and every single new post to it. But, if it ever does get to be too much, you can mark all as read and/or separate the different categories of notifications from one another - community posts by others vs. replies to your own content.

    PieFed really is leaving Lemmy behind in the dust, as far as features are concerned.

  • mesa@piefed.socialEnglish
    4·
    9 months ago

    Maybe search by tags? I know piefed allows you to do that. And Mastodon.

    Either that or make your own search via api and exclude any community with more than X subscribers. Probably would work well.

    • PumpkinDrama@reddthat.comBanned from communityOPEnglish
      1·
      9 months ago

      I’m fine just using the hot sort or scaled sort in my subscribed communities. I don’t participate much since I mainly post in niche communities, and Lemmy tends to smother those, so I have to keep using Reddit.

  • other_cat@piefed.zipEnglish
    3·
    9 months ago

    I do Top Day for my subscribed communities, and then when I get through that, I switch to All and Active and I’ve found quite a few interesting posts and communities that way.

  • podbrushkin@mander.xyzEnglish
    3·
    9 months ago

    Instead of small communities we have a billion posts per second about Trump and Charlie Kirk. It’s not possible to hide them, filtered keywords feature doesn’t work for pictures. Also, have you seen latest anti-MAGA meme? It’s fire, go check it out, just switch your feed to Global.

      • podbrushkin@mander.xyzEnglish
        2·
        9 months ago

        Nah, it’s like brooming in the forest. If only keywords worked for pictures… (in pic next post is about trump btw)

    • Skavau@piefed.socialEnglish
      9·
      9 months ago

      You realise if Lemmy/Piefed even doubled in size, /new/ would just be useless. Just a wave of low quality posts, spam, and topically non-relevant posts to most people.

      It doesn’t scale.

        • Skavau@piefed.socialEnglish
          2·
          9 months ago

          But that also happens with the Hot and Popular filters.

          I use /hot/ for local and subscribed posts after not looking at them for a day or so sometimes.

          If someone could only look by /new/ on the /all/ anything older than an hour or so would just be completely gone unless they kept scrolling. The feed would be irrelevant to most people or dominated by frequent posters who flood their communities. In fact, it would make it more desirable for community owners to flood communities full of low-effort posts as the only way to attain visibility would be to be ever-present on the /new/ feed.

          Because if I just swipe through the All option across all the federated instances using the Hot or Popular filters it’s mostly memes, circlejerking and rageposting. Is this what is wanted? Because nobody needs Lemmy for that. Reddit and all of the other platforms are already serving that low level of engagement with slop to spare.

          Why do you imagine a mandatory sort by /new/ would be less likely to be a feed of memes, circlejerking and rageposting?

          This actively rewards it. Exactly the same way.

          As I said: Only /new/ existing would make it more desirable for community owners to flood communities full of low-effort posts as the only way to attain visibility would be to be ever-present on the /new/ feed.

          I don’t know what you want out of lemmy, but if that outcome is what you intend to have, the end product will have virtually no distinction of its’ counterparts and people will see no reason to change. As there will be none outside of it’s not “corporate owned”. But if it’s still slop, people who want a change, will just skip the whole thing. As I did for a while. And if it wasn’t for Piefed, I don’t think I would be still hanging around here anymore either.

          A platforming being ‘slop’ won’t be any less slop purely because it removes its /hot/ feed. In fact, you might as well just outright remove upvotes and downvotes at that rate. And then it just isn’t a reddit alternative anymore.

          I have not used those filters in a long time. But I’m betting that the enraged posting about what’s happening in the U.S. (and justifiably so), shitposting, circlejerking, memes and Tankies causing controversy again are taking the entire feed until one gets bored. If I’m wrong I bet is not by much. But hell, I could’ve stayed on reddit for that back when I was there which was a while ago.

          You can just outright remove all of the ‘tankie’ instances from your own viewing if you want. Especially on Piefed. You can block all the shitpost and circlejerk and meme communities.

          Meanwhile the communities with intellectually engaging posting, real propositions of solutions and thoughtful discussions get slided to nowhere like on every other platform. They’re here and they are incredible. But guess what, they’re also on reddit, youtube, instagram and so on. And they get even more traction than in here. And also no traction in comparison to everywhere else. Just like here. But they do still get more people than here in the end.

          What communities are you referring to here?

          So what is the appeal of changing?

          Well I came here because I wanted to run a particular community that I couldn’t run or help on reddit. Reddit has exhausted itself for people who want to community build. Almost all names are taken.

          There are many other issues with Reddit too: people being able to hide their post history (thus making it much easier for bad faith accounts to hide their posting history), no voting visibility (I didn’t know the Fediverse had this before I joined, but it’s very good in that it cultivates a high-trust culture), a broken block system, and its beginning to administrate via AI tools meaning people are getting their posts hidden or removed based on its poor understanding. On the Fediverse you can actually directly interact with instance owners and admins, making each instance much more accountable to users - and if you don’t like how one community is run in one instance, you can create it elsewhere and take their users (if enough people agree).

            • Skavau@piefed.socialEnglish
              1·
              9 months ago

              But to be fair I didn’t say that killing those filters would solve every problem, just that it would eliminate that implicit incentive to seek the reward to be highlighted at the top. Through “easy to be enraged by posts”, “karma-farming like reddit posts” or just hoards of low effort memes just to mention a few.

              There is no karma-posting on here though. People post a lot because they want their communities to grow. This is the case with /new/ only or /hot/. The habits won’t change.

              Mentioning the upvote/downvote system, I always said that we could do without it and it would be for the better. I said it on reddit for many years. I say it here as well. It’s actually the most easy to hijack feature of them all: bots, brigade hits, you name it. I never thought of this being the defining feature of reddit, but its biggest flaw. The community building, the interchangeable branches of discussions, that was what reddit got right and was stolen to the end of the earth for it. Virtually every platform stole this after 2005/2006.

              Remember though that the Fediverse has public voting, so any attempts to game it are often caught and the perpetrators banned. This is very unlike Reddit. Upvoting/downvoting as a system is not perfect, it’s incomplete - and alternative systems could exist, however a react system of some sort is necessary to curate content across the board for the audience.

              Like I said, I have the list of communities as my homepage and then I go through them chronologically in whatever I’m interested to check out that day. I don’t know what you mean by never seeing any posts past an hour. I don’t follow any communities here that if I don’t come here for three or four days (which happens frequently) that I couldn’t go through them quickly. Even back then on reddit, I didn’t have this issue with the communities that I followed. But I never followed high traffic nonsense.

              I mean that if only /new/ existed, then after an hour or so - a small community is invisible unless they keep posting constantly. Because people are unlikely to scroll back and notice posts from that community.

              We should have like a “town hall” like instance for all federated administrators and mods to talk and vote on directions to take and have users vote. This is only valuable if we implement the same rails and safeguards as in the structuring of a great democracy. And Lemmy can function like a beacon to why federated municipality should be the future of democracy. Even deciding a cap on the number of maximum registrations per instance is not a bad idea to start throwing as numbers increase. I was on Lemm.ee. It was nothing terrible that happened there, it was just too much.

              The Fediverse is far too small to even discuss bringing up guards against growth now. In addition, instances can have wildly different - and do have wildly different local policies on community creation, account creation, federation, upvoting (some disable upvotes and downvotes) and many other things if possible. Some instances - as I’m sure you are aware of just don’t get on with each other and their own respective userbases would have different opinions over policy.

              It’s better to have large numbers spread around many instances, than to have them in just a few. This is the right way to scale up. The more centralised and large the more easy it is to corrupt it. And the harder it is to manage it. Smaller instances will know better how to maintain its base and to manage it. Rimu (the piefed creator) already said he is thinking about closing the registrations on the instance he manages. Which is the one I’m on.

              I must have missed Rimu saying that recently. I am also on piefed.social. He only communciated that for storage reasons to me in the past.

              By all means help people set up other instances, but have them be small. Lemmy.world is constantly being hit with new registered trolls and instigators and it is very clear why they pick it, because it is the larger instance and hijacks the lemmyverse attention in the process through this same filters we are speaking of, which is whay I’m on this detour. I also want to say that the administrators and mods have done a great job, but I’m seeing the same cracks that I saw on lemm.ee start to show in that is becoming too much for them to handle so much. It’s not that they are making bad decisions or becoming terrible at all, is that it is starting to become impossible for them to keep the standards they have maintained so far. It’s too much to handle. That’s all.

              This is part because the Lemmy software lacks trivial safeguards against day 1 trolls by the way. They can’t even delay community creation for X days for new accounts because Lemmy simply doesn’t have it built in as an option.

    • Scrubbles@poptalk.scrubbles.techEnglish
      6·
      9 months ago

      Algorithm is not a dirty word. Any sorting or filtering is an algorithm. Just because you don’t like something doesn’t mean we need to kill it off, just don’t use them.

    • ShaggySnacks@lemmy.myserv.oneEnglish
      2·
      9 months ago

      PS: Also, Crossposting obviously helps smaller instances and communities to get noticed too.

      Honsetly, crossposting is an amazing feature. The vast majority of my posts are just me crossposting posts to another community. It’s low effort-high value. It takes a few minutes to bring new content to smaller communities.

  • Tracaine@lemmy.worldBannedEnglish
    56·
    9 months ago

    Compete with what? Maybe the small communities should just not suck.

    • Chozo@fedia.io
      3·
      9 months ago

      It’s a signal-to-noise issue. There are some smaller communities I’d like to keep an eye on, but the posts from those communities get drowned out by the more active ones. I miss a lot of posts that I would have liked to have seen because of this.