And yet, China is using AI.
…I… don’t know what to think about that.
…I really don’t.
Because it seems that AI is just a scam.
It may “exist” but what it can do is a scam.
Maybe China thinks we have to use it just to “keep up” with the Western powers, but I dunno.
Anyway, interesting discussion with Adam Conover and Ed Zitron. It’s long, but you can listen to it while doing other things. And the comments are interesting too, but then again, there are also trolls in the comments as well (AI supporters here and there).
Frankly, though? I oppose AI. I’m anti-AI. I’m anti-AI in China and anti-AI in America and anti-AI in the whole damn planet.
I would argue that if your goal is to get an output that is the statistical mean for a given input, then sure an LLM will generate a set of outputs that statistically go together. It just happens that you throw enough data at it and waste a small country’s annual energy consumption then of course you’ll get something statistically similar. Congrats. You did it.
The energy consumption has already been reduced drastically by reinforcement learning, mixture of agents, quantizing, and other techniques. We’re literally just starting to optimize this tech, and there’s already been huge progress in that regard. Second, it’s already quite good at doing real world tasks, and saves me a ton of time writing boilerplate when coding.
So, that’s another thing that I wonder about. All these LLMs are doing is automating boilerplate code, and frankly that’s not really innovative. “Programming via stack overflow” was a joke that has been in use for nearly two decades now (shudder) and all the LLM is doing is saving you the ALT+TAB between SO and your text editor, no?
If you’re doing a TODO app in Angular or NextJS I’m sure you get tons of boilerplate.
But what about when it comes to novel, original work? How much does that help? I mean really how much savings do you get, and how useful was it?
The reality is that most of programming isn’t all that innovative. Most work in general isn’t innovative. Automating the boring stuff is literally the whole point. Meanwhile, it’s far more sophisticated than copy pasting from StackOverflow. It can come up with solutions in a context of a specific problem you give it, and the chain of reasoning DeepSeek R1 produces is actually interesting in itself, as it reads like a chain of thought.
This itself can actually be useful for doing novel and original work because it stimulates your thinking. Sometimes you see something that triggers an idea you wouldn’t have had otherwise, and you can pull on this thread. I find it literally saves me hours of work, and it is very useful.
For example, just the other day I used it to come up with a SQL table schema based on some sample input JSON data. Figuring out the relationships would’ve taken me a little while, and then typing it all up even longer. It did exactly what I needed, and let me focus on the problem I wanted to solve. I also find it can be useful for analyzing code which is great for getting introduced to a codebase you’re not familiar with, or finding a specific part of the code that might be of interest.
It’s also able to find places in code that can be optimized and even write the optimizations itself https://github.com/ggml-org/llama.cpp/pull/11453
Based on my experience, I can definitively says that his is a genuinely useful too for software development.
How likely is it that this JSON structure and corresponding database schema is somewhere in the (immense) training data. Was it novel? New?
Like I just have continual questions about an LLM doing 3NF a priori on novel data.
Like if we just outright say that LLMs are just a better Google or a better IntelliSense that can fetch you existing data that it has seen (which, given that it’s basically the entire Internet, across probably the entire existence of the Internet that has been crawled by crawlers and the Internet archive, which is a boggling amount) instead of dressing it up as coming up with NEW AND ENTIRELY NOVEL code like the hype keeps saying, then I’d be less of a detractor
You seem to think that the way these things work is by just simply pulling up chunks of existing code from a big db. That’s not actually what’s happening. It’s building a solution for a particular context based on its training data. This is not fundamentally different from how a human developer solves problems either. You learn through experience and once you’ve solved many problems, you recognize patterns and apply solutions you learned previously in a new context. It is writing new and novel code when it produces solutions.
Sorry, that is not what I think. It’s just that surely there was something very similar enough to your JSON to get the prediction to come up with something that looks similar enough. It’s very annoying having to discuss an LLMs intricate details of how it works and then get nitpicked on a concept that I don’t think I was saying
That’s the thing it’s not that there was something similar to my specific problem. It’s parts of many billions of problems that have been fed into the system have aspects that all come together to produce a solution for the particular problem. You don’t even have to start with JSON, you can ask it to come up a JSON schema by simply giving it parameters.
I’m not nitpicking you, I’m pointing out that what you’ve described isn’t actually how these things worked, and the way they do work is far more useful in practice. Let me give you a concrete example. Given a query of “write a json schema to represent a comment thread on a social media site like reddit”, it’ll do this bit of reasoning:
Okay, I need to create a JSON schema for a comment thread similar to Reddit. Let me start by breaking down what a Reddit-style comment thread includes. First, each comment has an ID, author info, content, timestamps, votes, and replies. The replies are nested comments, forming a tree structure. So the schema should allow for nested objects within each comment's replies. I need to define the required fields for a comment. The ID is essential, as is the content and author information. The author should have a username, maybe a profile link, and flags like is_mod or is_admin. Voting information: upvotes and downvotes counts, and a total score. Also, whether the current user has voted, which could be an enum indicating up, down, or none. Timestamps for when the comment was created and last edited. Maybe include an edited field that's either a boolean or a timestamp. Replies need to be an array of comments, which means the schema will reference itself. That requires using "$ref" to point to the comment definition, allowing for recursion. Moderation aspects: flags for removed, locked, spam, etc. Distinguish between user deletion and mod removal. User mentions and links: comments might reference other users or contain URLs, so including arrays for those. Awards or badges on comments, similar to Reddit's awards. Each award could have a name, image URL, and count. Pagination for long threads? Maybe include a "continue_thread" field with a token for loading more replies. Sorting options for replies (best, top, new, etc.), so a sort_order field with enum values. Let me structure the schema with definitions. Start with the comment object, then define author, award, user mention, link, etc. Required fields: id, author, content, created_at, score. Maybe others depending on necessity. For the voting, current_user_vote should be an enum of "up", "down", or null. Ensure that all optional fields are not required. For example, edited can be a boolean or a timestamp, but maybe just a timestamp if it's been edited. Validation: each comment's replies must be an array of comment objects. Use "items" with "$ref" to the comment schema. Testing edge cases: deleted users, removed comments, maximum nesting depth. But the schema might not handle depth limits, just the structure. Include examples to illustrate how a comment with replies would look. I think that covers the main aspects. Now, draft the schema with these considerations.This alone is already useful because it goes through the steps of the problem, identifies some corner cases, and suggests what a good way to model this problem might be. It will then produce a json schema based on that:
{ "$schema": "http://json-schema.org/draft-07/schema", "title": "Social Media Comment Thread", "type": "object", "definitions": { "comment": { "type": "object", "properties": { "id": { "type": "string", "format": "uuid", "description": "Unique identifier for the comment" }, "author": { "$ref": "#/definitions/user" }, "content": { "type": "string", "description": "Markdown-formatted comment content" }, "created_at": { "type": "string", "format": "date-time" }, ... "total_comments": { "type": "integer", "minimum": 0 }, "depth_limit": { "type": "integer", "description": "Maximum allowed comment depth" }, "comment_limit": { "type": "integer", "description": "Maximum comments per page" } }, "required": ["thread"] }it highlights key features of the solution
Key Features: Nested Comments: Recursive structure for replies Voting System: Track upvotes/downvotes and user's vote User Identity: Detailed author information Moderation: Flags for removed/locked/deleted content Awards: Support for Reddit-style awards Mentions & Links: Track user mentions and embedded links Sorting & Pagination: Handle different sort orders and partial threads Safety Features: Distinguish user deletion vs mod removal This schema supports: Infinite nesting of comments Multiple sorting algorithms User reputation systems Content moderation tools Rich text formatting Pagination for large threads User interaction trackingand it gives an example usage (I’ve truncated it for brevity)
{ "thread": { "id": "550e8400-e29b-41d4-a716-446655440000", "author": { "username": "tech_fan42", "avatar": "https://example.com/avatar.jpg", "is_mod": false }, "content": "This new framework is amazing! 🚀", "created_at": "2023-08-20T14:30:00Z", "score": 1542, "replies": [ { "id": "3f3e6c60-3d7d-461d-aa7a-5c94f5d5dc5d", "author": { "username": "framework_author", "is_admin": true, "distinguished": "admin" }, "content": "Thanks! Check out our [docs](https://docs.example.com/)", "created_at": "2023-08-20T14:35:00Z", "score": 893, "links": [ { "url": "https://docs.example.com/", "text": "docs" } ] } ], "awards": [ { "name": "Gold", "image_url": "https://example.com/gold.png", "count": 2 } ] }, "total_comments": 45, "depth_limit": 8 }Now, given that schema, I can get it to produce a SQL schema from that, write the queries to do operations against the DB, etc. So, I can go directly from getting requirements in plain English, to having a reasonable data model, and a persistence layer in a span of a few minutes. I can also ask it to explain design decisions, make adjustments, etc.
This is far closer to pair programming with a second developer than hunting for code to glue together on Stack Overflow.