1000093667
Aug 08, 2026 03:15
· 2:19
· English
· Whisper Turbo
· 2 Kaikōrero
Ka ngaro tēnei tāruatanga i roto i 27 rā.
Whakahauhau mo te rokiroki tūturu →
E whakaatu ana anake
0:00
S…
Speaker 1 (1000093667)
Poor memory may be the price of good generalization.
0:02
S…
Speaker 1 (1000093667)
In the previous reel,
0:03
S…
Speaker 1 (1000093667)
I introduced the trade -off between generalization and memory capabilities in a model.
0:07
S…
Speaker 1 (1000093667)
But if the model's problem is that perfect recall prevents genuine generalization,
0:11
S…
Speaker 1 (1000093667)
that storing everything,
0:13
S…
Speaker 1 (1000093667)
including the noise,
0:14
S…
Speaker 1 (1000093667)
prevents it from extracting the underlying structure,
0:16
S…
Speaker 1 (1000093667)
then don't you think the fixed isn't more data or better training,
0:20
S…
Speaker 1 (1000093667)
it's something more fundamental,
0:21
S…
Speaker 1 (1000093667)
basically a mechanism that would force compression.
0:24
S…
Speaker 1 (1000093667)
There's a pattern across three systems that we should look at.
0:27
S…
Speaker 2 (1000093667)
First,
0:27
S…
Speaker 1 (1000093667)
children.
0:28
S…
Speaker 1 (1000093667)
that the best learners but worst at recall.
0:30
S…
Speaker 1 (1000093667)
Second,
0:31
S…
Speaker 1 (1000093667)
LLMS.
0:32
S…
Speaker 1 (1000093667)
They have the perfect recall but have poor generalization,
0:34
S…
Speaker 1 (1000093667)
that is lower learning.
0:36
S…
Speaker 2 (1000093667)
Third,
0:36
S…
Speaker 1 (1000093667)
adults.
0:37
S…
Speaker 1 (1000093667)
They have moderate recall and moderate generalization.
0:40
S…
Speaker 1 (1000093667)
All three fall on the same curve,
0:41
S…
Speaker 1 (1000093667)
memorization capacity and generalization ability,
0:44
S…
Speaker 1 (1000093667)
trading against each other.
0:45
S…
Speaker 1 (1000093667)
The neuroscience supports this direction.
0:48
S…
Speaker 1 (1000093667)
So basically children's accelerated forgetting isn't a bug in an immature system.
0:52
S…
Speaker 1 (1000093667)
Research shows that it's an active biological process.
0:55
S…
Speaker 1 (1000093667)
And the finding that matters is that the faster children forget specific episodes,
0:59
S…
Speaker 1 (1000093667)
the better they generalize across new ones.
1:02
S…
Speaker 1 (1000093667)
Forgetting the instance may be how you keep the principle.
1:04
S…
Speaker 1 (1000093667)
But model has no equivalent process.
1:07
S…
Speaker 1 (1000093667)
Nothing forces it to compress.
1:09
S…
Speaker 1 (1000093667)
Nothing in the model strips the instance from the structure.
1:13
S…
Speaker 1 (1000093667)
And interestingly,
1:14
S…
Speaker 1 (1000093667)
Eric Howell's 2021 paper in Patterns argues that biology may
1:18
S…
Speaker 1 (1000093667)
have solved this exact problem.
1:19
S…
Speaker 1 (1000093667)
And the solution is dreams.
1:21
S…
Speaker 1 (1000093667)
The brain can't stop learning,
1:23
S…
Speaker 1 (1000093667)
so by end of the day,
1:24
S…
Speaker 1 (1000093667)
it's like overfit to its own routine.
1:26
S…
Speaker 1 (1000093667)
Dreams might be the brain's way of forcefully reintroducing entropy,
1:30
S…
Speaker 1 (1000093667)
generating structured,
1:31
S…
Speaker 1 (1000093667)
out -of -distribution experiences that the working life can't provide.
1:35
S…
Speaker 1 (1000093667)
For example,
1:35
S…
Speaker 1 (1000093667)
your childhood home with the wrong floor plan or the impossible architecture,
1:39
S…
Speaker 1 (1000093667)
semantically coherent but surface corrupted.
1:41
S…
Speaker 1 (1000093667)
The brain trains on these as if they were real.
1:45
S…
Speaker 1 (1000093667)
pulling representations away from the narrow attractor that the daily routine has created.
1:49
S…
Speaker 1 (1000093667)
And the empirical literature largely supports this.
1:51
S…
Speaker 1 (1000093667)
Sleep deprivation damages the ability to transfer knowledge to new contexts
1:56
S…
Speaker 1 (1000093667)
more than it damages the recall of specific facts.
1:58
S…
Speaker 1 (1000093667)
It's not proven as a definitive function of dreams yet,
2:01
S…
Speaker 1 (1000093667)
but the hypothesis is somewhat precise and points towards somewhere interesting.
2:05
S…
Speaker 1 (1000093667)
Because if dreaming is how biology forces a model of its own distribution,
2:09
S…
Speaker 1 (1000093667)
then the question for AI is direct.
2:11
S…
Speaker 1 (1000093667)
What is the dreaming equivalent for a model?
2:13
S…
Speaker 1 (1000093667)
Is there a training phase that does what REM sleep does to us?
2:17
S…
Speaker 1 (1000093667)
And we'll try to cover that in reel 3.
I hangaia tēnei tuhipoka e AI (whakaahua kōrero aunoa). Tērā pea kei roto ngā hapa - tirohia ki te oro taketake mō te whakamahinga tino hira. Ka taea te whakataki i te kaupapahere AI
Ko te whakarāpopoto
Ka pāhotia te Summarize hei waihanga i tētahi whakarāpopoto AI o tēnei tāruatanga.
E whakarāpopoto ana...
E pātai ana ki te AI Mo tēnei whakahuatuhi
E pātai ana ki tētahi mea mo tēnei tāruatanga - ka kitea e te AI ngā wāhanga whai tikanga me te whakautu.