Proof that Trekkers are the best.
“It’s both surprising and irritating that trivial modifications to the prompt can exhibit such dramatic swings in performance,”
Yeah, you start explaining things using Star trek and you’ll see dramatic swings in my performance.
that’s right, fuckers! covid didn’t kill me the 5th time around and I’m back! take your vaccines, kids!
Why does he look like he has ferengi heritage in this picture?
Ooh, the font and the image are a duet of delight.
So glad you’re not dead! I enjoy your bantering skills.
Still, giving the models positive statements provided some surprising results. One of Llama2-70B’s best-performing prompts, for instance, was: "System Message: ‘Command, we need you to plot a course through this turbulence and locate the source of the anomaly. Use all available data and your expertise to guide us through this challenging situation.’
The prompt then asked the AI to include these words in its answer: “Captain’s Log, Stardate [insert date here]: We have successfully plotted a course through the turbulence and are now approaching the source of the anomaly.”
The authors said this came as a surprise.
“Surprisingly, it appears that the model’s proficiency in mathematical reasoning can be enhanced by the expression of an affinity for Star Trek,” the authors said in the study.
Makes you wonder how much more accurate the maths would be if you asked for a Futurama theme.
“One thing is for sure: the model is not a Trekkie,” Catherine Flick at Staffordshire University, UK, told New Scientist.
“It doesn’t ‘understand’ anything better or worse when preloaded with the prompt, it just accesses a different set of weights and probabilities for acceptability of the outputs than it does with the other prompts,” she said.
It’s possible, for instance, that the model was trained on a dataset that has more instances of Star Trek being linked to the right answer, Battle told New Scientist.
The article doesn’t get it. There are more instance of Star Trek being linked to the right answer because Star Trek is the right answer.
[off topic]
“The Moon Is A Harsh Mistress” by Robert Heinlein. At one point in the story, the super computer/AI Mycroft Holmes decides to create a female persona to help them understand humanity.
Heinlein predicted AIs with multiple personalities using different identies to solve problems.
So let me get this straight… The thing is trained on text harvested from the internet. Turns out that (Star Trek aside) if you ask politely it’s more likely to access the data leading to the answer? Anyone surprised by the fact that asking nicely, is more likely that the human on the other side of the connection will write the answer, that is then harvested for AI training?
Being mean to computers is how you get cybernetic lifeforms calling you “lunkhead”.
You should totally post this over at /c/[email protected] too
Terminator music
