1000093667

Aug 08, 2026 03:15 · 2:19 · English · Whisper Turbo · 2 Speakers
This transcript expires in 27 dagar. Uppgradering til permanent lagring →
Lærarabókin
Showing only
0:00
S… Speaker 1 (1000093667)
Poor memory may be the price of good generalization.
0:02
S… Speaker 1 (1000093667)
In the previous reel,
0:03
S… Speaker 1 (1000093667)
I introduced the trade -off between generalization and memory capabilities in a model.
0:07
S… Speaker 1 (1000093667)
But if the model's problem is that perfect recall prevents genuine generalization,
0:11
S… Speaker 1 (1000093667)
that storing everything,
0:13
S… Speaker 1 (1000093667)
including the noise,
0:14
S… Speaker 1 (1000093667)
prevents it from extracting the underlying structure,
0:16
S… Speaker 1 (1000093667)
then don't you think the fixed isn't more data or better training,
0:20
S… Speaker 1 (1000093667)
it's something more fundamental,
0:21
S… Speaker 1 (1000093667)
basically a mechanism that would force compression.
0:24
S… Speaker 1 (1000093667)
There's a pattern across three systems that we should look at.
0:27
S… Speaker 2 (1000093667)
First,
0:27
S… Speaker 1 (1000093667)
children.
0:28
S… Speaker 1 (1000093667)
that the best learners but worst at recall.
0:30
S… Speaker 1 (1000093667)
Second,
0:31
S… Speaker 1 (1000093667)
LLMS.
0:32
S… Speaker 1 (1000093667)
They have the perfect recall but have poor generalization,
0:34
S… Speaker 1 (1000093667)
that is lower learning.
0:36
S… Speaker 2 (1000093667)
Third,
0:36
S… Speaker 1 (1000093667)
adults.
0:37
S… Speaker 1 (1000093667)
They have moderate recall and moderate generalization.
0:40
S… Speaker 1 (1000093667)
All three fall on the same curve,
0:41
S… Speaker 1 (1000093667)
memorization capacity and generalization ability,
0:44
S… Speaker 1 (1000093667)
trading against each other.
0:45
S… Speaker 1 (1000093667)
The neuroscience supports this direction.
0:48
S… Speaker 1 (1000093667)
So basically children's accelerated forgetting isn't a bug in an immature system.
0:52
S… Speaker 1 (1000093667)
Research shows that it's an active biological process.
0:55
S… Speaker 1 (1000093667)
And the finding that matters is that the faster children forget specific episodes,
0:59
S… Speaker 1 (1000093667)
the better they generalize across new ones.
1:02
S… Speaker 1 (1000093667)
Forgetting the instance may be how you keep the principle.
1:04
S… Speaker 1 (1000093667)
But model has no equivalent process.
1:07
S… Speaker 1 (1000093667)
Nothing forces it to compress.
1:09
S… Speaker 1 (1000093667)
Nothing in the model strips the instance from the structure.
1:13
S… Speaker 1 (1000093667)
And interestingly,
1:14
S… Speaker 1 (1000093667)
Eric Howell's 2021 paper in Patterns argues that biology may
1:18
S… Speaker 1 (1000093667)
have solved this exact problem.
1:19
S… Speaker 1 (1000093667)
And the solution is dreams.
1:21
S… Speaker 1 (1000093667)
The brain can't stop learning,
1:23
S… Speaker 1 (1000093667)
so by end of the day,
1:24
S… Speaker 1 (1000093667)
it's like overfit to its own routine.
1:26
S… Speaker 1 (1000093667)
Dreams might be the brain's way of forcefully reintroducing entropy,
1:30
S… Speaker 1 (1000093667)
generating structured,
1:31
S… Speaker 1 (1000093667)
out -of -distribution experiences that the working life can't provide.
1:35
S… Speaker 1 (1000093667)
For example,
1:35
S… Speaker 1 (1000093667)
your childhood home with the wrong floor plan or the impossible architecture,
1:39
S… Speaker 1 (1000093667)
semantically coherent but surface corrupted.
1:41
S… Speaker 1 (1000093667)
The brain trains on these as if they were real.
1:45
S… Speaker 1 (1000093667)
pulling representations away from the narrow attractor that the daily routine has created.
1:49
S… Speaker 1 (1000093667)
And the empirical literature largely supports this.
1:51
S… Speaker 1 (1000093667)
Sleep deprivation damages the ability to transfer knowledge to new contexts
1:56
S… Speaker 1 (1000093667)
more than it damages the recall of specific facts.
1:58
S… Speaker 1 (1000093667)
It's not proven as a definitive function of dreams yet,
2:01
S… Speaker 1 (1000093667)
but the hypothesis is somewhat precise and points towards somewhere interesting.
2:05
S… Speaker 1 (1000093667)
Because if dreaming is how biology forces a model of its own distribution,
2:09
S… Speaker 1 (1000093667)
then the question for AI is direct.
2:11
S… Speaker 1 (1000093667)
What is the dreaming equivalent for a model?
2:13
S… Speaker 1 (1000093667)
Is there a training phase that does what REM sleep does to us?
2:17
S… Speaker 1 (1000093667)
And we'll try to cover that in reel 3.

This transcript was generated by AI (automatic speech recognition). May contain errors - check against the original audio for critical use. Politikkur

❤️ Love STT.ai? Tell your friends!
Samandráttur
Click Summarize to generate an AI summary of this transcript.
Summating...
Spørg AI um hetta transkriptión
Tað er ein spurningur um, hvussu nógvar ferðir ein kann skriva og lesa.
Transcribe otro