They support Claude, ChatGPT, Gemini, HuggingChat, and Mistral.

  • nu11@sh.itjust.worksEnglish
    9·
    3 hours ago

    I don’t understand the hate. It’s just a sidebar for the supported LLMs. Maybe I’m misunderstanding?

    Yes, I would prefer Mozilla focus on the browser, but to me, this seems like it was done in an afternoon.

  • ocassionallyaduck@lemmy.world
    16·
    8 hours ago

    Thing is, for your average user with no GPU and whp never thinks about RAM, running a local LLM is intimidating. But it shouldn’t be. Any system with an integrated GPU, and the more RAM the better, can run simple models locally.

    The not so dirty secret is that ChatGPT 3 vs 4 isn’t that big a difference, and neither are leaps and bounds ahead of the publically available models for about 99% of tasks. For that 1% people will ooh and aah over it, but 99% of use cases are only seeing marginal gains on 4o.

    And the simplified models that run “only” 95% as well? They can use 90% fewer resources give pretty much identical answers outside of hyperspecific use cases.

    Running a a “smol” model as some are called, gets you all the bang for none of the buck, and your data stays on your system and never leaves.

    I’ve been yelling from the rooftops to some stupid corporate types that once the model is trained, it’s trained. Unless you are training models yourself, there is no need for the massive AI clusters, just for the model. Run it local on your hardware at a fraction of the cost.

    • LWD@lemm.ee
      15·
      6 hours ago

      There’s the tragedy with this new feature: they fast-tracked this past more popular requests, sticking it into Release Firefox.

      But they only rushed the part that connects to third parties. There was also a “localhost” option which was originally alongside the Big Five corporate offerings, but Mozilla ultimately decided to bury that one inside of the about:config settings.

      • MrOtherGuy@lemmy.world
        7·
        5 hours ago

        I’m guessing that the reason (and a good one at that) is that simply having an option to connect to a local chatbot leads to just confused users because they also need the actual chatbot running on their system. If you can set up that, then you can certainly toggle a simple switch in about:config to show the option.

    • ilhamagh@lemmy.world
      3·
      5 hours ago

      Can you point me to some resources to running smol llm?

      My use case prob just to help “typing” miscellaneous idea I have or check for my grammatical error, in english.

      Thanks, in advance.

    • Lojcs@lemm.ee
      21·
      4 hours ago

      Last time I tried using a local llm (about a year ago) it generated only a couple words per second and the answers were barely relevant. Also I don’t see how a local llm can fulfill the glorified search engine role that people use llms for.

      • TheDorkfromYork@lemm.eeEnglish
        1·
        4 hours ago

        They’re fast and high quality now. ChatGPT is the best, but local llms are great, even with 10gb of vram.

  • onlooker@lemmy.ml
    82·
    6 hours ago

    For a second I thought it said “experimental failure”. Would be more accurate, I think.

  • marcie (she/her)@lemmy.ml
    104·
    7 hours ago

    why a fucking chatbot? translate a page better for me you fucking losers, all the translation options suck for privacy outside of specifically trained local AIs. this is the BEST use case for a small local LLM yet mozilla with all its brains and resources couldnt rub two neurons together for this.

    or they could do character prediction on your typing to make typing faster. just some legit examples, why waste resources to build a chat ai into my browser when i can just open a website???

    • Midnitte@beehaw.orgEnglish
      31·
      6 hours ago

      Perhaps Mozilla’s biggest “failure” is just communication…

      Firefox actually has this now.

  • Eiri@lemmy.ca
    15·
    11 hours ago

    I wish I had telemetry on such features.

    I really doubt a significant number of people use AI chatbots often enough that having it in a dedicated sidebar is worth it.

    • Possibly linuxEnglish
      149·
      10 hours ago

      I wish I had telemetry

      I’m sure they do as Mozilla is an ad company

    • thingsiplay@beehaw.org
      81·
      9 hours ago

      There are no open source ai models, even if they tell you that they are. HuggingFace is the closest thing to as something like open source where you can download ai models to run locally without internet connection. There are applications for that. In Firefox the HuggingChat uses models from HuggingFace, but I think it is running them on a server and does not download from?

      The reason why they are not open source is, because we don’t know exactly on what data they are trained on. We cannot rebuild them on our own. And for trustworthy, I assume you are talking about the integration and the software using the models, right? At least it is implemented by Mozilla, so there is (to me) some sort of trust involved. Yes, even after all the bullshit I trust Mozilla.

      • chicken@lemmy.dbzer0.com
        2·
        6 hours ago

        It’s “open weights” if they are publishing the model file but nothing about its creation. There’s some hypothetical security concerns with training it to give very specific outputs for certain very specific inputs but I feel like that’s one of those kind of far fetched worries especially if you want to use it for chat or summarization and the comparison is getting AI output from a server API. Local is still way better.

    • 1rre@discuss.tchncs.de
      10·
      13 hours ago

      I think Mistral is model-available (ie I’m not sure if they release training data/code but they do release model shape and weights), huggingchat definitely is open source and model-available