Whisper is a popular transcription tool powered by artificial intelligence, but it has a major flaw. It makes things up that were never said.

cross-posted from: https://lemmy.world/post/21301373

Tech behemoth OpenAI has touted its artificial intelligence-powered transcription tool Whisper as having near “human level robustness and accuracy.”

But Whisper has a major flaw: It is prone to making up chunks of text or even entire sentences, according to interviews with more than a dozen software engineers, developers and academic researchers. Those experts said some of the invented text — known in the industry as hallucinations — can include racial commentary, violent rhetoric and even imagined medical treatments.

Experts said that such fabrications are problematic because Whisper is being used in a slew of industries worldwide to translate and transcribe interviews, generate text in popular consumer technologies and create subtitles for videos.

More concerning, they said, is a rush by medical centers to utilize Whisper-based tools to transcribe patients’ consultations with doctors, despite OpenAI’ s warnings that the tool should not be used in “high-risk domains.”

  • JimmyBigSausage@lemm.ee
    191·
    26 days ago

    This AI thing isn’t working. Apple’s new AI tool advancement (investors are panting in anticipation) is a cartoon drawing tool? This has been around awhile now just online.

    • halcyoncmdr@lemmy.worldEnglish
      5·
      25 days ago

      Traditional voice recognition systems are… Okay from my experience. As soon as you get loud background sounds, speaking over each other, or large variations in speaking volume they often start to breakdown.

      Whisper works a lot better than the other systems I’ve tried. I’ve used Whisper to automate generating subtitles for videos, with great results, even with quiet phrasing and with background audio.

      I haven’t seen it hallucinate personally with anything I’ve thrown it at, but my sample size is also pretty small compared to something like an entire healthcare system.

      • conciselyverbose@sh.itjust.works
        5·
        25 days ago

        Pre-process. Noise reduction algorithms work really well, and that’s way more in line with how your brain is actually processing stimuli anyways.

    • Flying Squid@lemmy.world
      61·
      25 days ago

      Fucking why?

      “We bought all these server farms for AI and we have to use them for something!”