I’ve gone down a rabbit hole here.
I’ve been looking at lk99, the potential room temp superconductor, lately. Then I came across an AI chat and decided to test it. I then asked it to propose a room temp superconductor and it suggested (NdBaCaCuO)_7(SrCuO_2)_2 and a means of production which got me thinking. It’s just a system for looking at patterns and answering the question. I’m not saying this has made anything new, but it seems to me eventually a chat AI would be able to suggest a new material fairly easily.
Has AI actually discovered or invented anything outside of it’s own computer industry and how close are we to it doing stuff humans haven’t done before?
It’s important to be clear what kind of actual system you’re using when you say “AI”.
If you’re talking about something like ChatGPT, you’re using an LLM, or “Large Language Model”. Its goal is to produce something that reasonably looks like a human wrote it. It has reviewed a ridiculous amount of human text, and has a metric assload of weights associating the relationships between these words.
If the LLM sees your question and associates a particular compound with superconductors, it’s because it’s seen these things related in other writings (directly or indirectly).
It’s important not to ascribe more intent behind what your seeing than exists. It can’t understand what a superconductor is or how materials can achieve the state, it’s just really good at relaying related words in a convincing manner
That’s not to say it isn’t cool or useful, or that ML(Machine Learning) can’t be used to help find answers to these kinds of questions.
Exactly. It’s just text prediction software that is really good at making itself sound plausible. It could tell you something completely false and have no idea it’s stating a lie. There’s no intelligence here. It’s a very precise word guesser. Which is great for specific settings. But there’s a huge amount of hype associated with this tool and it’s very much by design (by tech companies).
If the LLM sees your question and associates a particular compound with superconductors, it’s because it’s seen these things related in other writings (directly or indirectly).
I’m not convinced of this. LLMs haven’t been just spitting out prior art, despite what some people seem to suggest. It’s not just auto-complete, that’s just a useful analogy.
For instance, I’m fascinated by the study that got GPT4 to draw a unicorn using LaTeX. It wasn’t great, but it was recognizable to us as a unicorn. And apparently that’s gotten better with iterations. GPT (presumably) has no idea what a unicorn looks like, except through text descriptions. Who knows how it goes from written descriptions of a mythical being to a 2d drawing with a markup language without being trained on images, imagery, or any concept of what things look like.
It’s important not to ascribe more intent behind what your seeing than exists.
But also, this is true as well. I’m trying hard not to anthropomorphize this LLM but it sure seems like there’s some emergent effect that kind of looks like an intelligence to a layman like myself.
To be clear, I’m not trying to make the argument that it can only produce exactly what it’s seen, I recognize that this argument is frankly overstated in media. (The interviews with Adam Conover are great examples; he’s not wrong per se, but he does oversimplify things to the point that I think a lot of people misunderstand what’s being discussed)
The ability to recombine what it’s seen in different ways as an emergent property is interesting and provocative, but isn’t really what OP is asking about.
A better example of how LLMs can be useful in research like what OP described would be asking it to coalesce information from multiple existing studies about what properties correlate with superconducting in order to help accelerate research in collaboration with actual material scientists. This is all research that could be done without LLMs, or even without ML, but having a general way to parse and filter these kinds of documents is still incredibly powerful, and will be a sort of force multiplication for these researchers going forward.
My favorite example of the limitation on LLM’s is to ask it to coin a new word, then google that word. It physically is unable to produce a combination of letters that it doesn’t have indexed, and it doesn’t have an index for words it hasn’t seen. It might be able to create a new meaning for a word that it’s seen, but that isn’t necessarily the same.
Define AI.
Machine learning, image processing and similar “AI” techniques have been used for decades in sciences. When a telescope does large field survey to detect an transient phenomena like a supernova, you don’t have an astrophysicist looking at the photo, a smart astrophysicist (well several ones) used image processing and machine learning to teach the computer that there is something interesting on the image and send an automated message to every telescope about : Something is happening at this position so they can watch it immediately. Is it AI ?
What about the LHC who takes so much data that they need very advanced algorithm to store them and process them, is that AI ? What about protein folding which is a very complex problem relying on machine learning to find the proper solution
i believe google created something to do with protein folding or something using deepmind
idk tho
Alphafold2 Pretty cool tech that I used in my degree to solve an impossible test posed by my tutor.
The fact that this thing can get so close to x-ray crystallography results is just amazing.
AI is a super broad topic, I’ve heard people refer to Principal Component Analysis, which is from the 1930s, as “Machine Learning” or “AI”. In reality its just that we have infrastructure and data at scale to start applying techniques in larger contexts.
I know pharmaceutical companies have been using AI in drug discovery for probably a decade now, but those models are very different from what a large language model looks like, and you still have a human sifting through the results and performing validation on a physical system to make sure the compounds do what is predicted safely, most of which do not.
When you ask something like ChatGPT a question like that, its doing something akin to looking up the most recent papers on the subject that it was trained on, and outputting something that looks like a chemical compound that the paper would be generated. It doesn’t have an understanding of what that formula means, only that when you arrange letters in that way, it looks superficially similar to what would have been in the paper. Its like in movies, when they need to express someone doing math, they just fill a chalk board with random equations that look like advanced math at first glance, but might be introductory level material, or even just gibberish.
You wouldn’t use an LLM (chat bot to appear human) for finding new materials, you’d specialize it with physical and chemical data and give it technically requirements. Something similar is done via AI for genome sequencing. AI is also used to design complex structures that are hard for humans to get their heads behind, like a new structure of a CPU. AI can also create new art, but depending on the model, similar to how humans do it, it will take more inspiration or less from already existing works. AI is not completely new, as we’ve used machine learning to filter stuff via pattern or finding anomalies for decades. It just never was this easy to train your own model and generate usable results.
The AIs we have at our disposal can’t invent a thing - yet - because they aren’t true AIs - again: yet.
They are merely, and should be perceived as tools, nothing more. It’s the people who use them that may apply them to tasks that will result in invention, but on their own, they are closer to the Chinese Room principle, than to thinking and inventive constructions.
I agree with the basic idea, but there’s not some fundamental distinction between what we have now and true AI. Maybe we’ll find breakthroughs that help, but the systems we’re using now would work given enough computing power and training. There’s nothing the human brain can do that they can’t, so with enough resources they can imitate the human brain.
Making one smarter than a human wouldn’t be completely trivial, but I doubt it would be all that difficult given that the AI is powerful enough to imitate something smarter than a human.
A real AI would have explained you that with great probability a room temperature (and atmospheric pressure) superconductor is not possible.
Few experiments were successful with small grains tested under enormous pressures. But apart from that a room temperature superconductor is unlikely due to the high entropy.
Google LK99
Yes, AI has and is being used extensively for research already, particularly in problem spaces where pattern matching can yield particularly powerful results for solution searches, which is actually a lot of problem spaces. Protein structures are probably the best example.
The AI most regular people are using, you can think of as better at sifting the sea of data to find what you are looking for, and discussing it with you in a more understandable way.
The breakthroughs with AI are often computer programs that are told to look for an answer and change themselves somewhat randomly, but with many, many copies trying the task. The ones that fail are deleted, and the most successful change up again. This is repeated until you reliably get your result.
So basically darwinism?
pretty much, but much faster.
The “breakthrough” likely happened a long time ago, but as with all tech, it was only recently accessible to the general public. LLM alone isn’t even that sophisticated to begin with.
AI assistant will soon be an essential part of our lives. It will handle grocery shopping based on your dietary requirements, conduct basic diagnosis of your health, create personalized software, books, music, and movies on the fly for you, do your taxes and offer financial advices. All these are already happening.
I assume you are referring to transformers, which came out in the literature around 2017. Attention on its own is significantly older, but wasn’t really used in a context that came close to being used as a large language model until the early / mid 2010s.
While attention is fairly simplistic, a trait which helps it parallelize well and scale well, there is a lot of research that came about recently around how the text is presented to the model, and the size of the models. There is also a lot of sophistication around instruction tuning and alignment as well which is how you get from simple text continuation to something that can answer questions. I don’t think you could make something like chatGPT using just the 2017 “Attention is All You Need” paper.
I suspect that publicly released models lags whatever google or OpenAI has figured out by 6 months to a year, especially because there is now a lot of shareholder pressure around releasing LLM based products. Advancement that are developed in the open source community, like apply LoRA and quantization in various contexts, has a significantly shorter time between development and release.
