An Observation on Generalization

Video quality	The size	Download

Information An Observation on Generalization

Title	:	An Observation on Generalization
Lasting	:	57.21
Date of publication	:
Views	:	167 rb

Sicophantic crowd itrying/i to laugh is about the worst sound a group of people can make He's smart, he's not a god, stfu and listen!
Comment from : @kob8634

Search 'An Observation on Generalization' from Simons to get the better version of this video
Comment from : @RunDaChansey

:( we need more updates on the Safe Superintelligence Inc Initiative :(((((((((((
Comment from : @AiExplicado0001

This was fantastic I actually understand these concepts now
Comment from : @englishredneckintexas6604

This was very shortly before the OpenAI crisis when he tried to get altman fired
Comment from : @winsomehax

Great talk!
Comment from : @cc98-oe7ol

I miss him
Comment from : @digzrow8745

great talk !
Comment from : @于磊-w5p

Great talk!! Thanks for publishing
Comment from : @baboothewonderspam

Great presentation!
Comment from : @bboysil

The linear property itself is a compression, this is a trivial observation that can be obtained by studying differential manifolds
Comment from : @xyh6552

Compression can explain the part of unsupervised learning that is very similar to the operation of using the regularity lemma in mathematics to deal with graph theory problems Essentially, it utilizes the almost orthogonality of some local graphs
Comment from : @xyh6552

it's going to be great success according to ilya
Comment from : @consumidorbrasileiro222

Huh, just explain normal
Comment from : @RoboticusMusic

42:50 Is that Scott Aaronson?
Comment from : @AnirudhAjith

Within the first minute it is admitted that "OpenAI" is not open Open development doesn't have aspects that developers "can't talk about" By 20 minutes in, the guy in the video has said thousands of words without saying almost anything at all, it's like he's winging it, or doesn't really understand his subjectbrbrThe guy in the video even admits machine learning is not particularly difficult to understand, yet he does a terrible job of conveying understanding It's almost like he's more interested in trying to convey the impression he's really intelligent, rather than trying to convey meaning and understandingbrbrI suppose it's possible that English is his second language, or perhaps he's just not that experienced at public speaking Maybe he's one of those of people who's really good with mathematics, but terrible with language? I just get the impression from this video that he's something of a charlatan
Comment from : @TPQ1980

This is pathetic what is even generalization
Comment from : @Wubbay828

"A particular observation on generalization" would've been a cooler title 🙃
Comment from : @rv706

OpenAI is now worth 90B The employees have sold their shares to investors They have their roi after 7 yrs
Comment from : @dotnet364

Ilya is so good that I have been forced to buy $MSFT to secure my UBI after my white collar job gets replaced by AI:)
Comment from : @aburaziel

What is data compression?
Comment from : @joeremus9039

How is this guy regarded one of the best experts in AI is beyond me His level of intuitive understanding of neural networks is depressing
Comment from : @krultorwaru121

Is that Scott Aaronson asking questions in the audience :D?
Comment from : @Jonathan-k3r8r

Binary tokens
Comment from : @jameelbarnes3458

What a beard, Ilya⁉️ Suits yahh! So what am I going to learn this time❓️😁😇🌹
Comment from : @adtiamzon3663

In global workspace theory, there is bottleneck representation After this talk, I strongly believe bottleneck is feature, because bottleneck forces model to learn nice compression way, which we call inductive reasoning
Comment from : @kimchi_taco

Information theory to the rescue of unsupervised learning, who would have thought ;)
Comment from : @deepbayes6808

So is he saying there is some algorithm to take compression of say Large Language Models and generalize them to work with say vision?
Comment from : @joeysipos

Interesting talk about compression, thanks!
Comment from : @Morimea

If anyone is wondering, the equations for statistical learning theory for supervised learning were drawn from his phd thesis: wwwcsutorontoca/ilya/pubs/ilya_sutskever_phd_thesispdf
Comment from : @deepbuilds4553

A avatar "@Yuksel Mert Cankus" in the chat nailed it (or at least nailed a good comment on the topic) Because in most practical use cases we can tolerate errors, and because a sentient mind is the final user of a NN, it might not always be so useful to focus on achieving a numerical approximation to Kolmogorov compression There is a profoundly fuzzy goal: find an Ai tool that's useable, better than yesterday's tool, and compute efficientbr (Let's be honest and note the utopian goal of sentient subjective awareness, aka "consciousness" and uploading your mind into silicon is laughable If you end up doing it, you can come back to now in your time machine and tell me all about it)
Comment from : @Achrononmaster

what's the reference for the inequality he showed at around 4:05?
Comment from : @huyle3597

An interesting exploration of a lot of these points (including Solomonoff induction) is presented in the paper "Playing games with AIs: The limits of GPT-3 and similar Large Language Models" -- highly recommended reading
Comment from : @EdedML

Scott Aaronson asking question!?
Comment from : @vev

In the phenomenology of consciousness qualia are great compressors
Comment from : @kenmogibrainworld4844

👍
Comment from : @IAjayMukhiya

I think your intro about supervised learning is a bit missleading Yes it is true theoretically that test-errortraining-error if the model complexety is small, for instance in terms of VC-dimension brHowever, this VC theory cannot explain the success of deep learning In practice, large models with billions of parameter are used and trained to nearly 0 training-error and STILL the test error is small Applying the VC-dimension argument to these models doesn't give you anything because the observed test-error is magnitudes times smaller than the predicted test-error from the VC-theory
Comment from : @jonatan8392

It will be interesting in the future to see how the compression of AI systems gets better and better They have already had neural networks discover physical laws in raw experimental data, but how little data is needed for that? Nobody knows! Maybe it just needs eg a few seconds of airwaves broad spectrum signal This also makes any safety concept for oracle AGIs that involves somehow keeping "safety critical data" from the AGI a very weak approach The world is highly correlated, and a higher intelligence might learn much more from what we give it, than we see in it ourselves
Comment from : @tristanwegner

Hear me out: unsupervised learning on synthetic data Look up Code-lama-unnatural benchmarks
Comment from : @nanow1990

2 minutes in, and I can already tell this talk is going to be fascinating
Comment from : @odomobo

Ray Solomonoff's version of compression is very similar to Kolmogorov's, and Solomonoff's version of induction is predicting the next token Another random Ray thing, in the 1980s he was worried that a smart enough supervised AI would effectively learn the trainer's objective function and turn the tables
Comment from : @NerdFuture

The compression/prediction could be more simply thought of as pattern recognition
Comment from : @eskelCz

Ilya Sutskver da god
Comment from : @wrathofgrothendieck

All on top of the "SGD miracle" When we realise that SGD is just crap, all this DL horseshit will fall into oblivion
Comment from : @JohnDoe-nv2op

내가 가장 좋아하는 대머리!!
Comment from : @specifictoken

Thank you
Comment from : @dreamphoenix

Talk starts at 0:14
Comment from : @afrozenator

S tier brain D tier PowerPoint aesthetics
Comment from : @markpfeffer7487

Reminds me of VR concerts
Comment from : @alonsomartinez9588

It makes sense
Comment from : @shinkurt

Excellent talk
Comment from : @simonstrandgaard5503

50:00 the size of the compressor (GPT4) is a salient term in the inequality 😂
Comment from : @binjianxin7830

Ilya is great / sympatic I hope some day you can communicate and exchange some trade secrets so its not too much and not too little Because a 100 secrecy seems counterproductive as well as a 100 openness might lose to people just hiding everything and copycats or even hypothetical worse tactics So much for proprietary information :)
Comment from : @hanskraut2018

Supervised learning is essentially an artifice Unsupervised learning is the foundation of Intelligence Kohonen was the true pioneer in this, and everything since seems to be extensions and tweaked variations on his foundational ideas
Comment from : @rohanfernando

The guy at the end gets at the point that the semantics are scrambled and these guys are confused about semantics and the expression of the problems Obviously very smart but bad at writing about this stufff/expressing it
Comment from : @syphiliticpangloss

Why is he calling "next pixel prediction" and UNSUPERVISED task?
Comment from : @syphiliticpangloss

well well, brwe'll see what it all comes down to brblah blah blah so far brwe will see the effects in the real world, brnot on the board
Comment from : @GraczPierwszy

It's all about entropy and complexity
Comment from : @kenwolf887

What are 100 videos that are MUST-WATCH for AI enthusiasts in 2023?
Comment from : @SchoolofAI

The eloquence of his delivery was delightful
Comment from : @mamotivated

31:10 GPT models _JC
Comment from : @JCResDoc94

he is high on what?
Comment from : @Alejandro388

ClosedAI
Comment from : @Sarmadpervez186

We need more people that are at least half as smart as him
Comment from : @jimlbeaver

Does anyone know the name of the theorem being shown at 4:54 so that I can look it up to learn?
Comment from : @jony7779

thx so much for uploading from Germany!
Comment from : @lukas-santopuglisi668

Why is your company called OpenAI if you're the opposite of open? OpenAI is supposedly the "leader" of the AI space, yet it's the least generous out of all AI companies Meta is constantly releasing new research, tools, open-source software and the rest, while OpenAI is busy playing politics and trying to squash competition It's really quite sad and disappointing No wonder you had such a large exodus of talent Hopefully you'll change your ways
Comment from : @WalterSamuels