I was wondering if the search engines are indexing the fediverse. Because recently Google search became so bad so I was always putting Reddit at the end to get some not so generic results and I haven’t even noticed to get any replies pointing to the fediverse ever. So I was wondering if search engines were actually indexing the content here?

  • kadu@lemmy.world
    link
    fedilink
    arrow-up
    29
    ·
    1 year ago

    Yes, but keep in mind top level domains like .world .ai .life and similar are heavily punished by the search algorithm.

    So let’s say you search for “best Android apps 2023” and there’s a Lemmy.world post with this exact title and a great list of apps, versus a clickbait techblog written by ChatGPT and a .com domain… The techblog will very likely be ranked higher.

    • mim@lemmy.sdf.org
      link
      fedilink
      arrow-up
      8
      ·
      1 year ago

      One good thing about Reddit is that one could add a “site:reddit.com” at the end of each Google search to improve the results.

      This won’t work as well with Lemmy (since it’s all a bunch of different domains). Hope that in the future a feature like this is implemented to search across the fediverse

      • aesopjah@lemm.ee
        link
        fedilink
        arrow-up
        7
        ·
        1 year ago

        Yeah, perhaps another qualifier that’s, say, specifies the protocol, e.g. Activity Pub? Would be handy

  • OutrageousUmpire@lemmy.world
    link
    fedilink
    arrow-up
    3
    ·
    1 year ago

    Sure they are. But the amount of info in the fediverse right now is minuscule to say R*ddit. It will take time and growth before we start seeing fediverse stuff show up regularly in search.

  • octt@feddit.it
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    1 year ago

    Some platforms like Mastodon allow individual users to discourage search engines from indexing their profile. But, by default, as far as I know all platforms allow indexing. Lemmy seems to not provide any option to control this kind of thing so everything should get indexed.

    The thing is, using “thingtosearch reddit” you’re not using any search engine properly, that’s kind of a hack. What you would do is actually “thingtosearch site:reddit.com” to limit searches to a specific site. This works with any site, of course, so you could for example do “thingtosearch site:feddit.it” (that’s my instance), and you will get specific results (which actually might include results from other instances, due to how this indexing works, even though they will be displayed from the site of the instance you specified). (I just noticed btw that DuckDuckGo doesn’t list anything for site:myinstance… well, that’s strange, Google has no problem.)

    • cmat273@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      DuckDuckGo uses their own web crawlers along with a few others (but not google’s). Probably just haven’t indexed it yet.