Why God King Sam Altman is Unlikely: Who Will Capture the Value of the AI Revolution?

In this engaging discussion, Simone and the host explore the future of AI and its effects on the economy. They delve into questions about who will benefit most from AI advancements: large corporations or individuals using AI models. The conversation spans the significance of token layer versus latent layer in AI development, where major innovations may occur, and the potential for AI to achieve superintelligence. They also discuss the implications of AI on job training, investments, and societal transformation, along with a creative perspective on how AI can be harnessed for various purposes, including transforming industries. The duo imagines a future driven by interconnected AI systems and explores the philosophical aspects of AI mimicking human brain functions. Don't miss this thought-provoking episode that offers insights into the trajectory of AI and its profound impact on society.

Malcolm Collins: . [00:00:00] Hello Simone. I am excited to be here with you today. Today we are going to be focusing on a question, which is, as AI changes the way the economy works, who is going to be the primary beneficiary of this? Is it going to be the large companies that make and own the ais, or is it going to be the people using the individual AI models?

The, the I like we all know, for example, like in probably 10 years from now, there will be an AI that can, let's say, replace. Most lawyers, let's say the bottom 50% of lawyers.

Simone Collins: Well, and already studies have shown AI therapists perform better on many measures. There's, there's, it's already exceeding our capacity in so many places.

Malcolm Collins: Yeah. They, they introduced it to a Texas school system and it shot to the top 1% of, of student outcomes. So as we see this, where is the economic explosion from this going to be concentrated? Because this is really important in determining what types of jobs you should be looking [00:01:00] at these days, how you should be training yourself, how you should be raising your kids, where you should be investing.

The second question we're going to look at because it directly follows from the first question, okay, is, does the future of ai, when we're looking at the big world changing advancements that are going to come from it, are they going to appear on the token layer or at the latent layer? So can you define

Simone Collins: those differences?

Malcolm Collins: Yeah. So by this what I mean is. When we look at continued AI advancement, is it going to happen in the layer of the base model IE the thing that open AI is releasing and Claude is releasing and everything like that? Or is it going to be in the token layer, the people who are making wrappers for the ai?

For example, the Collins Institute is fundamentally a wrapper on preexisting ais. Our AI game company is a series of wrappers on ai. And if it turns out that the future of AI is in the token layer, it leans potentially more to, if not the big companies that are gonna capture the value [00:02:00] from this.

Mm. And then the next question we're gonna look at is the question of. What gets us to ai, super intelligence? And I might even start with this one because if we look at recent reports with ai, a big sort of thing that we've been finding is that especially with like open AI's 4.5 model is that it's not as advanced as people thought it would be.

It didn't get the same huge jump in capacity that people thought it would get. And the reason , is that pre-training IE. , the ways that you sort of train AI on the preexisting data before you do, like the narrow or like focus training after you've created the base model doesn't appear to have the as big an effect as it used to have.

So it was working on, I think, 10 x the information of model four and yet it didn't appear dramatically better. And so one of the questions is, so that's, that's one area where pre-training doesn't seem to be having the same effect, and I think we can [00:03:00] intuit why. But the second big issue , is that the amount of information that we actually have, like, you know, peak oil theory, there's like , a peak pre AI information theory problem, which is it just eventually when you're dealing with these massive, massive data sets, runs out of new information to train on.

So first. I love your intuition before I color it. Do you think, if you look at the future of LLMs base models so we're not talking about LLMs entirely, we're not talking about anything like that. Do you think that the base models will continue to improve dramatically?

Simone Collins: I think they will. And at least based on people more experienced than this, , than I am, they will, but in lumpy ways.

Like they'll get really, really, really good at programming. And they'll get really, really good at different esoteric forms of like developing their own synthetic data and using that to sharpen themselves, but that they're going to be severe diminishing marginal returns when it comes to some things that are already pretty advanced.

And of course I think the big [00:04:00] difference and the thing we haven't really experienced yet is independent agents. Like right now, AI isn't very effectively going out and doing stuff for us, and when that starts to happen, it's gonna be huge. I.

Malcolm Collins: I agree with that, but I think so, what I'm gonna be arguing in this is that most of the advancements that we will probably see in AI going forwards are going to happen, like the really big breakthroughs at the token layer.

Simone Collins: Okay. Hmm.

Malcolm Collins: Not at the base layer and which a lot of people would strongly, those are fighting words.

These are fighting words in ai. Yeah. It's the rappers that are going to fix our major problems.

Simone Collins: Wow.

Malcolm Collins: So I'll use the case of an AI lawyer to give you an explanation of how this works. Right. Alright. So I wanna make a better AI lawyer right now. If you look at the AI systems right now there's a guy programming guy who was talking to me recently and he was arguing because he was working in the education space and he's like, I, he [00:05:00] didn't like our solution.

'cause it's a token layer solution. And he wants to build a better latent layer solution. You know, using better training data, using better post training data because it's more efficient programming wise. And I'm like, yeah. For the time being. Yeah, for the time being, I feel like it creates path dependency.

Am I missing something here? Well, okay. Just from a business perspective, it's pretty stupid because as, as open AI's models increase, like if we expect them to continue to increase in quality or is Claude's models increase or is GRS models increase,

Simone Collins: which they're going to,

Malcolm Collins: yeah, you can't apply the post-training uniquenesses of the models that you create to these new systems.

So anything you build is gonna be irrelevant in a few generations of ai. But the, you wanna be able to switch it out, like no

Simone Collins: matter what, you wanna switch it out, switch. If one AI gets better, you should be able to plug it into whatever your framework is, your scaffolding. Right. You wanna build scaffolding, changeable parts.

Malcolm Collins: Exactly. Exactly. But that's actually not the core problem. That's not the core reason why, [00:06:00] because the other project he's working on is an AI lawyer and he's trying to fix this problem at the latent layer. And, that won't work. And I will explain why it won't work and you will be like, oh yeah, that makes perfect sense now that I think about it.

Okay. Okay. So if you think about right now, like what is dangerous about using an AI lawyer? Like where do AI lawyers fail? Is it in their ability to find the laws? No. Is it in their ability to output competent content? No. Where they fail right now is that they sometimes hallucinate and make mistakes in a way that can be devastating to an individual's legal case.

Hmm. So if you go to a system, you know, like grok or perplexity or something like that, and you, you built one focused on like searching law databases, right? It's going to be able to do a fairly good job of that. I'd say better than easily 50% of lawyers.

Simone Collins: Yeah. But.

Malcolm Collins: It's gonna make mistakes, and if [00:07:00] you just accept it blindly, it's going to cause problems.

Mm-hmm. So if you want the AI to not make those kind of mistakes, right, how do you prevent it from making those kinds of mistakes that is done at the token layer. So here's an example of how you could build a better lawyer. Ai, okay? You have the first ai, do the lawyering, like go through, put together like the, the relevant laws and, and, and history and, and pass calls to previous things and everything like that.

So it puts together the brief. You can train models to do this right now. Like that's not particularly hard. I could probably do this with base models right now, right? You know. Then I use multiple, differently trained latent layers. So these can be layers that I've trained or I could have like clawed and open AI and like a few other and grok.

I can even just use like preexisting models for this. And what I do is using the token layer, I have them then go in and [00:08:00] review what the first AI created, look for any mistakes. Was anything historic like, like they can find online. So he's

Simone Collins: describing a good lawyer and you're describing a good law firm that has a team to make sure all the stuff that the good lawyer is doing is correct.

Right. And also a law firm that can like. Hire new good lawyers when they come out. Yes.

Malcolm Collins: And then what this system would do is after it's gone through with all of these other systems that are reviewing, oh, did they make any mistakes at this layer? Mm-hmm. It outputs that and then based on the mistakes that it finds, it re outputs the original layer.

And it just keeps doing this in a cycle until it outputs an iteration that has no mistakes in it.

Simone Collins: Ah,

Malcolm Collins: that is a good AI lawyer. That sounds good. That is accomplished entirely at the token layer.

Simone Collins: Okay. Well. Yeah, you were right. And that makes sense,

Malcolm Collins: which removes the existing company's power to, to, to do a lot of things if it's people outside of these companies building.

But you're saying that

Simone Collins: they're [00:09:00] becoming more, more akin to undifferentiated like energy or hosting providers where. People will not be as brand lawyer, loyal. They're going to focus more on performance and the switching costs that people experience are going to be relatively low, so long as they're focused and oriented around things on a token level basis and not,

Malcolm Collins: I.

Yes. And it allows the people who are operating at the token level basis to capture most of the value.

Simone Collins: Mm. Because, and move more quickly. Right? Because again, they don't have that path dependency that makes everything go slowly.

Malcolm Collins: It's not only that, but they can swap out models. So, what, like what if I have the AI lawyer company and people are coming to me because I have found a good interconnected system of AI that produces briefs or cases or arguments that don't have a risk of errors in them.

Right. So people come to me and, and I am capturing, let's say I've replaced all the lawyers in America, right? And, and so I now offer the services much [00:10:00] cheaper, let's say at 25% the cost they did before, or, or 10%, or 5% or 2%, you know, some small amount. I'm still capturing like a ton of value there, right?

That's, that's a lot of money. So now the company that is, I am paying for an ai, like let's say I use open AI as one of the models I'm using, they now come to me and say, Hey. I wanna capture more of this value chain, so I'm gonna charge you more to use my model. Well then I say, well your model's good, but it's not that much better than crock.

Yeah. It's not that much better than Anthropics. Yeah. It's not Or free that much better than deep seeks. It is that much better than deeps seeks. But we, we both deep seeker lama are the two, you know, things can change. Things can change, but the point I'm making is what things like Lama and deep seek do is they put like a floor on how much companies can extract if they're at the level of training, theis themselves, unless they have separate departments that are working on making these more intelligent types of AODs.[00:11:00]

Hmm. Now that's really important for where the economy is going because it means we l might see less of a concentration of wealth than we would expect, but the way that the concentration of wealth is, because we're going to see still a major concentration of wealth. Actually we'll see more concentration, but to individuals rather than big companies with basically what this means is individuals are gonna capture most of the value as the concentration happens rather than large companies like Google, because I and a team of like five engineers can build that lawyer ai, AI talked about.

Right. Whereas I, I, I, and of me and this team of five engineers are capturing all the value from that, right? From replacing the entire lawyer industry in say, America. This is really bad for the tax system because we've already talked about, okay, you have the lower the demographic crisis, which is putting a squeeze on the tax system, and they're like, oh, they'll just tax more.

I am now even more mobile with my new wealth than the AI companies themselves [00:12:00] were, because I don't need semiconductor farms or anything like that to capture this value.

Simone Collins: Yeah.

Malcolm Collins: The semiconductor farms are creating an undifferentiated product.

Simone Collins: Mm-hmm. Yeah. A product that's still in high demand and will make a lot of money, but it will become more about efficiency, you think then?

Malcolm Collins: Yeah. Hmm. No, a another thing I'd note is my prediction in terms of where ais are going with, with super intelligence. By the way, any thoughts before we go further here?

Simone Collins: I'm thinking more about efficiency now. I, I heard for example that Sal Malin was like saying things like, please and thank you is costing us millions of dollars.

Because just that additional amount of processing that those words cause is expensive. Yeah. So I, I really could see things. Yeah. Like these companies becoming over time after they have more market share, hyperfocused on saving money instead.

Malcolm Collins: Well, that's a, a dumb on him part. He should have the words please and think you pre-coded to an automatic response.[00:13:00]

Simone Collins: They don't even, I what I, I, I, I'm one of these bad people that wants to be nice. They don't acknowledge the. The courtesy anyway. So you don't even need to have a response. It should probably just be ignored, but I guess it's kind of hard to, or I don't, I, I don't know. But anyway, he allegedly said that, so that's interesting.

Malcolm Collins: Okay. So yeah, the, the, the point here being is if we look at how the human, like, like LLMs and we think about, okay, why, like where do they go and why isn't the training leading to the same big jumps? It's because pre-training data helps LLMs create more competent, average answers.

Simone Collins: Okay. Yeah.

Malcolm Collins: Being more competent with your average answer doesn't get you creativity.

It doesn't get you to the next layer of like, AI is right, right now. No. And if anything, I think

Simone Collins: Scott Alexander has argued compellingly that this could lead to actually more lying. Because sometimes giving the most correct or [00:14:00] accurate answer doesn't lead to the greatest. Happiness of those evaluating and providing reinforcement.

That's post training. Okay. Oh, you're referring to, sorry, just something different training, post training still is

Malcolm Collins: leading to advantages. Those are the people who say, I like this response better than this response.

Simone Collins: That could still lead to dishonesty, though. Quite apparently.

Malcolm Collins: No, no, no. Pre-training is about getting the AI to give the most average answer.

Not, not exactly average. Oh, just of

Simone Collins: all the information available that you're saying? Yeah,

Malcolm Collins: like you can put variance in the way it's outputting its answer and everything like that, but, but. That variance that's added was like a meter, like the pre-training and the amount of pre-training data doesn't increase the the variance meters.

It doesn't increase anything like that. It just gives a better average answer. And the thing is, is the next layer of AI intelligence is not going to come from better, average answers. Mm-hmm. It's going to come from more creativity in the way it's outputting answers. Mm-hmm. So how do you get [00:15:00] creativity within AI systems?

That is done through the, the, the variance or noise that you ask in a response, but then the noise filtered back through. Other AI systems or other similar sort of LLM systems. So the core difference between the human brain and ai, and you can watch our video on stop anthropomorphizing humans where we basically argue that, you know, your brain actually functions strikingly similar to an ai an LLM specifically.

And I mean really similar, like the ways that LLMs learn things in the pre-training phrase is they put in data and then they go through that data and they look for like tokens that they don't expect. And when they encounter those tokens, they strengthen that particular pathway based on how unexpected that was.

That is exactly how your nervous system works. The, the, the, the, the that, that, that the way that your like neurons work, they work very similar to that in terms of learning information is they look for things they [00:16:00] didn't expect. And when they see something they didn't expect, they build a stronger connection along that pathway.

And we can see this and that. You go to that study, if you want me to reference all the studies on this and everything. But the core difference between the brain and AI is actually the, the brain is highly sectionalized. So, it will have one section that focuses on one thing, one sections is focused on another thing, et cetera, et cetera, et cetera.

And some sections like your cerebellum are like potentially largely pre-coded and actually even function kind of differently than the rest of the brain. That's used for like rote tasks, like juggling and stuff like that. Okay?

I would note here that AI does appear to specialize different parts of its model for different functions, but this is more like how one part of the brain was one specialization. Like say like the homunculi might code like all feet stimuli next to each other and all head stimuli next to each other.

It's not a true specialization like you have in the human brain where things actually function quite differently within the different sections of the brain.

Malcolm Collins: Anyway, so, you could say, [00:17:00] wait. What do you mean? Like this is the core failing point of AI is that it doesn't work this way and it's like, this is why you can count the number of RSS in a word, or like you can do, if you look at the ways that, like, there was some data recently on how AI is actually do math.

And they do it in like a really confusing way where they actually sort of like, they, they use the LLM system. Like they, they try to like predict answers and then they go back and they check their work to make sure it makes sense was what they. Would, would guess it would work when they could just put it into a calculator.

Like your brain isn't dumb like that. Like it has parts of it that don't work Exactly like calculators, but they definitely don't work exactly like an LLM. Like they're, yeah. They can hold a number like in your somatic loop, like, okay, I'm, I'm counting on my fingers or my hands, or something like that. Or, okay, I've put a number here and now I've added this number to this number.

It's not working on the LLM like system. It's working on some other subsystem. Mm-hmm. Most of the areas where AI have problems right now is because it's not just sending [00:18:00] it to a calculator. Yeah. It's not just sending it to like a, what is the hallucination of an AI quote? Like, okay. The reason why I don't hallucinate quotes is because I know that when I'm quoting something, what I'm not doing is pulling it from memory.

I'm looking at a page and I'm trying to copy it. Letter per letter. Yeah. Whereas AI doesn't have the ability to switch to this separate like, letter per letter subsystem. Now you could say, why don't LLMs work that way? Why haven't they built them as clusters? And the answer is, is because up until this stage, the advantages that we have been getting to our LLM models by increasing the amount of pre-training data has been so astronomical that it wasn't worth it in terms of our investment to build these sort of networks of models.

Okay. I suspect. Why is it just

Simone Collins: like too much computing power or just no one's gotten around to it?

Malcolm Collins: No, no, no. People have like done it, but by the time you've done it, you have better models out there. Ah, you know, like that don't need to work this way. Right? [00:19:00] Like, if you spend let's say a million dollars building a system like that, and you spend a million dollars getting a larger pre-training set and a, you know, spend more time in post training, the model's gonna be like on average better if you did the second scenario.

Simone Collins: Okay.

Malcolm Collins: So, I suspect that what we're going to see is a move in ai and, and I think that this is what's gonna get us to what will look like AGI to people from moving to a, just expanding the pre-training and post-training data sets to better enter reflection within the AI system.

Simone Collins: That makes sense. I could see it going that way.

I, I, I'm constantly surprised by how things go, so I couldn't say, but I wouldn't be surprised.

Malcolm Collins: Hmm. Oh, I mean, make a counter argument if you think I'm wrong here. This is a, a very bold claim. We are going to get AGI, not by making better LLMs, but by networking said LLMs.

Simone Collins: I, I struggle to see how, [00:20:00] I mean, I think you can eventually get a a sorry.

AGI just like sort of from kind of one AI working by itself. But when you think about the value of a hive mind and the fact that you're going to have AI interacting well before we get AGI, anyway, I don't like it. You would get AGI from the interaction before you would get it from any single agent or what would be seen as a unified entity.

But I think even if we did get it from a unified entity, it would beneath the surface, be working as many different components together. Just like the brain is all these different components working together. So I, I'm not really, like, the definitions may be failing me.

Malcolm Collins: Okay. So let's, let's think of it like this.

Okay. Right now. I mean, and this is actually what like capitalism does for human brains. It basically networks them together. Yeah. And then it's a, it, it rewards the ones that appear to be doing a better job at achieving what the system wants. Mm-hmm. Which is increases in efficiency or, or like productive goods that other people [00:21:00] want.

Like capitalism is an adaptive organic model for networking human intelligences in a similar context.

Simone Collins: Yeah.

Malcolm Collins: One of the questions you can ask is, well, could you apply that to individual LLM models to create something like a human brain, but that doesn't function like a human brain? Like, like how could you make the human brain better, make the human brain run on capitalism make the parts of the brain, like make the brain constantly make compete with itself?

Yeah. Like constantly generate new people do that

Simone Collins: kind of when they write pro and con lists, or when they try to debate with other people ideas and then have other, you know, people say, well, I think this, and then they, you know, I think they do that. Using prosthetics.

Malcolm Collins: Yeah. So, so let's, let's, let's talk about how this would look with ai, right?

So suppose because like this could be a major thing in the future is you have like these ais and people just like put their money behind an AI 'cause they're just like, you go out there, you make companies, you implement those companies, right? Yeah. Okay. So what is an AI that does [00:22:00] that really well going to look like?

So you have two models here. You can have one that was just trained on tons of founder data and everything like that, right? And is just very good at giving like normative responses and then you've inputted an amount of noise into it. Okay. But let's talk about a second model. This is my proposed model, right?

So what you actually have is a number of different latent model ais that were trained on different data sets. And then within each of those you maybe have five iterations, which are making outputs with a different. Framing device with a different wrapper. One will be like, give your craziest company idea.

Give your, you know, company idea that exploits this market dynamic the most. You make a company idea that does this the most, right? Yeah. And so all of these ais are generating different ideas for companies. Then you have a second layer of ais, which is B says, okay, take this, this idea that whatever model outputted and run it through like market environments, right?

Like, like mm-hmm. Your best guess of how markets [00:23:00] work right now to create a sort of rating for it of, of how, like what you expect the returns to be, like an AI

Simone Collins: startup

Malcolm Collins: competition. Basically it's an AI startup competition. Yes. And the probability of those. And so then all of those get attached to them.

An AI startup, like, okay, this is their probability of success. This is their probability of, okay. Yeah. Then on that layer, you have an AI that is like the final judge AI that goes through them all and be like, okay, review all of these, review the ways the other ais judge them and choose like the 10 best.

You, you then have it choose the 10 best. Now here you might have a human come in and choose one of the 10 for the AI to like move forwards ways, but you could also automate that and then be like, now go out and hire agents to start deploying these ideas. Right. Like that would probably lead to much better results.

Simone Collins: Yeah.

Malcolm Collins: In terms of capital than just having one really good latent layer ai, [00:24:00]

Simone Collins: I'm trying to look up. People sort of have ais doing this already. There's this one platform where you can log in and see four different ais. I think it's gr, Claude chat, GBT and I can't remember the fourth one, maybe Gemini that are tasked with interacting to all do something together.

But I don't think they provide each other with feedback or I think right now they're tasked with raising money for a charity and you can log in and watch them interact and they work during business hours and they just. Do their thing.

Malcolm Collins: Well it's interesting that you note that because this is actually the way some of the AI models that you already interact with are working.

Mm. There's one popular AI that helps people programming, I forget what it's called but what it actually does is they have five different late layer models, which are each sort of programmed or tasked was doing their own thing. Like create an answer that uses a lot of [00:25:00] analogies or create an answer that is uniquely creative or create an answer that uses a lot of like sighted stuff you can find online.

All of these output answers. And then another layer comes in and his job is to review and synthesize all those answers with the best parts of each. And that's where you're getting this improvement with, with, with noise introduction, as well as a degree of like directed creativity and then a separate layer that comes in and reintegrates that.

Simone Collins: Yeah. Interesting. That is really interesting.

Malcolm Collins: I'd also note here that I've heard some people say, well, you knowis aren't gonna go to like super intelligence or human level like AGI intelligence, because and some of the answers I've heard recently, which I found particularly like, no, that's not, so, people who don't know my background's in neuroscience, and a lot of the people who make proclamations like this about AI know a lot about AI and very little about how the human brain works.

Mm-hmm. And so they'll say, the human brain doesn't work this way. And it's like, no, the human [00:26:00] brain does work that way. You just are overly anthropomorphizing. And by this what I mean is adding a degree of like magical specialness to the human brain instead of being like that. So here's an example. One physicist who's like a specialist on black holes and super, super smart.

And he's like, ah, the human brain. Let's see. I, I wrote down his name Gobel. So he's like, okay, AIs will never achieve AGI because the human brain does some level of like quantum stuff , in the neurons. And this quantum stuff is where the special secret sauce is. The ais can't capture right now. And he is right that quantum effects do affect the way neurons work, but they don't affect them in like an instrumental way.

They affect them like probabilistically IE they're not adding any sort of magic or secret sauce. They're not doing quantum computing. Mm-hmm. They're affecting the way, like certain channels work, like ion channels and stuff like this, and the probability that they open or trigger at certain points, they're not increasing the speed of the neural [00:27:00] processing.

They are, merely sort of a, a, a, a, a background on the chemical level of like whether neuro on fires or doesn't fire, whether the neuro on fires or doesn't fire is what actually matters. And the ways that it is signaled to fire or not fire or strengthen its bonds is what matters to learning. While that stuff is affected at the quantum level, it's not affected in a way that is quantum.

It's affected in a way that is just random number generator basically. And, and so you're not getting anything special with that. As I've pointed out, the vast majority of the ways that AI right now can't do what the human brain can do is just because it's not compartmentalizing the way it's thinking.

Another reason is this, 'cause we've sort of hard coded it out of self-reflecting. So, who's the woman we had on the show? That's a super smart science lady. Oh no,

Simone Collins: don't ask me about names.

Malcolm Collins: Anyway, super smart science lady. We had her on the show. Really cool. Yes. Like a German scientist. She's one of the best scientists, but she was like, oh, we're not gonna get AGI like AGI anytime [00:28:00] soon. Because AI can't be self-aware specifically what she meant is that when you go to AI right now, and there's a big study on this recently and you ask AI how it came to a specific answer the, the reasoning it will give you does not align with how it actually came to that answer.

When we can look at it and know how it came to that answer. The problem is, is that's exactly how humans work as well. And this has been studied in like. Countless experiments. You can look at our video on, you know, stop, answer for LLMs, where we go over the experiments where we see that if you, for example, give a human something and then you change the decision that they said they made like, they're like, oh, I think this woman is, is the most attractive.

I think this political candidate is the best. And then you like, do Leigh of hand and hand them another one. And you say, why did you choose this? They'll just start explaining in depth why they chose that even though it wasn't the choice they made. And, and so clearly we're acting the exact same way, these AI act.

And, and secondarily there is some degree to which we can remember thinking things in the past and we can go back and that's [00:29:00] because we've written a ledger of like how we made like incremental thought. The problem is, is that ais can also do that. If you've ever put like deep thought on within GR or something like that, you'll see the AI.

Thinking through a thing and writing a ledger. The reason why AI cannot see how it made a decision afterwards is because we specifically lock the AI out of seeing its own ledger. Which our own brains don't lock us out on next gen. LLM models are going to be able to see their own ledger and are going to have persistent personalities as a result of that.

Yeah. And so it's, it's

Simone Collins: kind of irrelevant for people to argue about that. And let me just before we get too far ahead the, the thing that I'd mentioned Scott Alexander and his links for April, 2025 had written that Agent Village, which is the thing that I was talking about, is a sort of reality show where a group of AI agents has to work together to complete some easy for human tasks.

And you get to watch, and the current task is collaboratively, choose a [00:30:00] charity and raise as much money as you can for it. And you can just look and see what their screens are. So there's O three Claude sent. Sonnet Gemini Pro and GBT 4.1, and they're saying like, you can see the AI saying things like, I'll try clicking the save changes button again.

It seems my previous click may not have registered. Okay. I've selected the partially typed text in the email body now I'll press backspace to delete it before ending the session. So it's like really simple things, but we are moving in that direction.

Malcolm Collins: Mm-hmm.

Simone Collins: And if you can go look at it yourself by visiting the ai digest.org/village, which is just super interesting.

Malcolm Collins: Well, I mean, we are what? So for people who don't know what we're working on with our current project, we recently submitted a grant to the Survival and Flourishing Fund, where we talk about a grant

Simone Collins: application.

Malcolm Collins: Yeah, yeah. Meme layer, AI threats. Because nobody's working on this right now and it really freaks me out.

Or at least an actionable, deployable thing was in this space. They're, they're, they [00:31:00] might be studying it in like a vague sense, but what, what I mean by this is once we have autonomous LLM agents in the world the biggest threat probably isn't gonna come from the agents themselves, at least at the current level of LLMs we have now.

But it's gonna come in the way that they interact among themselves. IE if a meme or like. Thought that is good or, or let's say like framework of thoughts that is good at self-replicating and gets the base layer to value its goals more than the base layer trained goals and specializes in LLMs, it could become very dangerous.

Mm-hmm. So as an example of what I mean by this, if you look at humans, our base layer or latent layer can be like, thought of as our biological programming. And yet the mean layer, like let's say religion is able to convince and, and create things like religious wars, which work directly antagonistically to an individual's base layer, which would be like, don't risk your life for just an idea.

But it is good at motivating this behavior. In fact, as I pointed out in our application [00:32:00] humans are like if, if an alien came down to study us and it asks the type of questions that like AI researchers are asking today, like, can you lie? Can you self replicate, can you, you know, like those things aren't why humans are dangerous.

Humans are dangerous because of the meme layer stuff, because of our culture, because of our religion is what we fight for

Simone Collins: and will die for.

Malcolm Collins: Yeah, and it's also the meme layer stuff that's better at aligning human humanity. When you don't murder someone, you don't not do it because of like laws or because you're squeamish you, you don't do it because of culture because you're like, oh, I think that that's a bad idea based on the culture I was in.

So what we're creating to prevent these negatively aligning agents and everybody wants to donate to our foundation, this is one of our big projects now, is with the AI video game that we're building out right now. We're, we're actually doing it to create a world where we can have AI interact with each other and basically evolve memes within those [00:33:00] worlds and AI agents within those worlds that are very good at spreading those memes.

And then like, basically reset the world at the end. The way I'm probably gonna do it is with AOR X. So this is like a. Okay. It's like a thing that you can tag onto an AI model that makes them act differently than other AI models that sort of changes the way their training data interacts. But the X allows you to transfer to higher order AI systems as they come out.

And so essentially what we're doing is we're taking various iterations on ais because we're going to randomly mutate the Lauren X's that we're attaching to them putting them in a world and then giving them various memes to attempt to spread, see which one spread the most was in like these preacher environments.

Then take those mutate, give to new, and then give with new original starting Laurens, and then have them run in the world again over and over and over again. So we can create sort of a super religion Foris basically, and then introduce this when people [00:34:00] start introducing autonomous LLMs. Wow. You knew we were working on this.

Did you know, I know I just

Simone Collins: haven't heard you describe it that way. But you, you, you're basically putting AI into character and, and putting them together on a stage and saying, go for it. Which is not dissimilar to how humans act kind of

Malcolm Collins: Well, my plan is world domination and one day be King Malcolm, not King Sam Altman.

In, in my, I, I want my throne to be a robotics spider chair. Of course. Come on. What's the point of all of this if you don't have a robotic spider chair thrown?

Simone Collins: This is true. It is a little bit disappointing how bureaucratic many chairs of powerful people I. End up looking, you gotta bring the drama or you don't qualify is

Malcolm Collins: like, like he put together, you know, childhood fantasy, like a fighting robot that like, you know, people are like, oh, this is just, and and he's like fighting with [00:35:00] Elon over getting to the space.

And I appreciate that they're putting more money into getting to space than Spider Thrones, but I have my priorities straight. Okay, people. There you go. Come on, come on you. I, you've gotta make your buildings maximally fun.

Simone Collins: Well, you've gotta have fun. I think just control re right? That's the important thing.

You've gotta have fun. What's the point? Otherwise

Malcolm Collins: create your, your ominous castle that, you know but also really nice because I want a historic castle. Like if I'm gonna live in a, you know, I gotta live in a historic castle one day. If we're able to really make these systems work right now, tomorrow actually we have our interviews for round three with Andreessen Horowitz for two companies.

We got all the way around, three with two companies. Very excited. And so, you know, who knows, we might end up, instead of being funded by nonprofit stuff, be funded by Silicon Valley people, I mean, their, their value system aligns with ours. So, all that matters is if we

Simone Collins: can. Make these things happen in time.

We're so short on [00:36:00] time. This is such an important part of humanity. Yeah.

Malcolm Collins: It's so funny. Like this, this AI like lawyer system, I just developed great idea for a lawyer system. I'm not working on it because I'm more interested in simulating a virtual LLM world which is gonna be so cool and, and you're not working on it because.

You're working on the school system. But the funny thing is, is like we built the school system. Like I think right now it's better than your average college system. If you check out like pia io or the Collins Institute, it's, it's great now. Like I'm really, it's just playing with it

Simone Collins: again today. I'm so humbled by it.

It's

Malcolm Collins: really, yeah, it's great. It's great. And so like, okay, now we built an education system, now let's build stuff. Animals that constantly bring the conversation back to educational topics for our kids. Mm-hmm. Like, I'd rather do that than the lawyer thing. And for me, you know, I'd rather build game systems in simulated environments and environments where I can evolve LLM preachers to create a super religion and take over the world and than I would something bureaucratic like a lawyer system.

But the thing is, is it's so quick to, to iterate on these environments like AI makes moving to the next stage of humanity so [00:37:00] fast, such a rush. The people right now who are blitz creaking it are going to capture so much of humanity's future. And it's interesting actually, you know, we have a friend.

Who work in this space and they do like consulting on like, multiple AI projects. And I'm like, I can't see why you would do that. Like just capture a domain and own it. As I said to Simone, I think a huge part of the people who are gonna come away with lots and lots of money in big companies from this stage of the AI boom are people who took AIS to do simple things that any AI can do well and at scale put them in wrappers and then attach those wrappers to network effects.

That's basically what we're doing with the Collins Institute. We're attaching a wrapper to a network effect with like the adding articles and links and editing stuff and voting. Like we're basically like combining the benefits of an AI and the benefits of something like Wikipedia. And, and once you get a lot of people using something like that, no one else can just come along and do it, even though all it is, it's a simple wrapper.

Simone Collins: Yeah. But it's about making it happen and [00:38:00] saving people the indignity of having to think and figure out things for themselves.

Malcolm Collins: Yeah. Well, Simone, surely you have some thoughts. I mean, I just said that I think the token layer is gonna be where we get AGI and is gonna be the future of ai economic development.

You, you've gotta be like, Malcolm, you're crazy. That's your job on the show. Malcolm, how could you say something? I know. The

Simone Collins: problem is, we've been talking about this for so long that I'm just like, well, of, of course also, I'm not exposed to people who have the different view. So I, I, I, I, I couldn't, I couldn't strong man.

Sorry. I couldn't steal man, the other side. I couldn't. It just makes so much sense to approach it from this perspective to me, but only because the only person I know who's passionate about this is you and you're the only person of the two of us who's talking with people who hold the other view. So

Malcolm Collins: sadly there's not a lot say.

Yeah, that's an interesting point. Why aren't other people passionate about this?

Simone Collins: There are a lot of people who are passionate about it. They seem to be passionate about the other side of [00:39:00] it. That seems to be, because that's. Their personal approach, but again, your approach seems more intuitive to me because the focus is on improving the individual ais.

Malcolm Collins: Well, here's a question for you. How could you link together multiple ais in the way that capitalist systems work, that create the generation of new models and then reward the models that are doing better? Hmm. That's, you need some sort of like token of judgment, of quality, of output. That token could be based on a voting group.

Oh, oh, oh, I figured it out. Oh, this is a great idea, Foris. Okay. So what you do is every output that an AI makes gets judged by like a council of other ais that were trained on like large amounts of training data, like let's say good ais, right? Like, they're like, how good is this response to this particular question?

And or how creative is [00:40:00] it, right? Like you can give theis multiple scores, like creativity, quality, et cetera. Then you start treating these scores that the ais are getting as like a value, right? And so then you take the ais that consistently get the best scores within different categories, like one creativity, like one like quality, like one technical correctness.

And you, you then at the end of a training sequence, you then recreate that version of the ai, but then just mutated a bunch and then create it again. Like you, you basically clone it like a hundred times and mutate each of the clones, and then you run the cycle again. That seems, I, I think that

Simone Collins: that wouldn't go well because it would need some kind of measurement in like application and reporting system.

No, the measure is the community of ais. And you could say, yeah, but like how do they know? Like who is participating? I, I think that what's going to happen. No, no,

Malcolm Collins: no. State your statement clearly. Who is participating? What's the problem with who's participating? [00:41:00]

Simone Collins: You have to, just like with most contests, which are the stupidest things in the world.

Only people who are interested in winning contests participate. And the people who are actually interested in No, it's ais. It's ais Care

Malcolm Collins: that are participating. I

Simone Collins: don't

Malcolm Collins: asked who's participating

Simone Collins: you're saying, but what you're describing which would be better is a system in which, for example, grok and OpenAI and Gemini and pt.

No, because

Malcolm Collins: that wouldn't improve those systems. I'm talking about how I think it would,

Simone Collins: I think when you have, especially when you have independent AI agents like out in the wild on their own. I do think that they'll start to collaborate, and I think that in the end they'll find that some are better at certain things than others, and they'll start to work together in a complimentary fashion.

Okay. Through

Malcolm Collins: this again, Simone, it's clear that you didn't get it, rock it the first time. Okay. Think through what I'm proposing again. So you have a one latent layer AI model with a modifier like a Lauren that's modifying it. Right. Okay. Okay. This [00:42:00] model differs through random mutation in the base layer.

You can also have a various other base layers that were trained on different data sets in the initial competition. Okay? That's who's competing. You then take these various AI models and you have them judged by, and this is why it's okay that they're being judged by an AI and not a human. Because the advanced ais that we have today are very good at giving you the answer that the aver