Home page
Telegram bot

An Observation on Generalization




Video quality The size Download

Information An Observation on Generalization


Title :  An Observation on Generalization
Lasting :   57.21
Date of publication :  
Views :   167 rb


Frames An Observation on Generalization





Description An Observation on Generalization



Comments An Observation on Generalization



@kob8634
Sicophantic crowd itrying/i to laugh is about the worst sound a group of people can make He's smart, he's not a god, stfu and listen!
Comment from : @kob8634


@RunDaChansey
Search 'An Observation on Generalization' from Simons to get the better version of this video
Comment from : @RunDaChansey


@AiExplicado0001
:( we need more updates on the Safe Superintelligence Inc Initiative :(((((((((((
Comment from : @AiExplicado0001


@englishredneckintexas6604
This was fantastic I actually understand these concepts now
Comment from : @englishredneckintexas6604


@winsomehax
This was very shortly before the OpenAI crisis when he tried to get altman fired
Comment from : @winsomehax


@cc98-oe7ol
Great talk!
Comment from : @cc98-oe7ol


@digzrow8745
I miss him
Comment from : @digzrow8745


@于磊-w5p
great talk !
Comment from : @于磊-w5p


@baboothewonderspam
Great talk!! Thanks for publishing
Comment from : @baboothewonderspam


@bboysil
Great presentation!
Comment from : @bboysil


@xyh6552
The linear property itself is a compression, this is a trivial observation that can be obtained by studying differential manifolds
Comment from : @xyh6552


@xyh6552
Compression can explain the part of unsupervised learning that is very similar to the operation of using the regularity lemma in mathematics to deal with graph theory problems Essentially, it utilizes the almost orthogonality of some local graphs
Comment from : @xyh6552


@consumidorbrasileiro222
it's going to be great success according to ilya
Comment from : @consumidorbrasileiro222


@RoboticusMusic
Huh, just explain normal
Comment from : @RoboticusMusic


@AnirudhAjith
42:50 Is that Scott Aaronson?
Comment from : @AnirudhAjith


@TPQ1980
Within the first minute it is admitted that "OpenAI" is not open Open development doesn't have aspects that developers "can't talk about" By 20 minutes in, the guy in the video has said thousands of words without saying almost anything at all, it's like he's winging it, or doesn't really understand his subjectbrbrThe guy in the video even admits machine learning is not particularly difficult to understand, yet he does a terrible job of conveying understanding It's almost like he's more interested in trying to convey the impression he's really intelligent, rather than trying to convey meaning and understandingbrbrI suppose it's possible that English is his second language, or perhaps he's just not that experienced at public speaking Maybe he's one of those of people who's really good with mathematics, but terrible with language? I just get the impression from this video that he's something of a charlatan
Comment from : @TPQ1980


@Wubbay828
This is pathetic what is even generalization
Comment from : @Wubbay828


@rv706
"A particular observation on generalization" would've been a cooler title 🙃
Comment from : @rv706


@dotnet364
OpenAI is now worth 90B The employees have sold their shares to investors They have their roi after 7 yrs
Comment from : @dotnet364


@aburaziel
Ilya is so good that I have been forced to buy $MSFT to secure my UBI after my white collar job gets replaced by AI:)
Comment from : @aburaziel


@joeremus9039
What is data compression?
Comment from : @joeremus9039


@krultorwaru121
How is this guy regarded one of the best experts in AI is beyond me His level of intuitive understanding of neural networks is depressing
Comment from : @krultorwaru121


@Jonathan-k3r8r
Is that Scott Aaronson asking questions in the audience :D?
Comment from : @Jonathan-k3r8r


@jameelbarnes3458
Binary tokens
Comment from : @jameelbarnes3458


@adtiamzon3663
What a beard, Ilya⁉️ Suits yahh! So what am I going to learn this time❓️😁😇🌹
Comment from : @adtiamzon3663


@kimchi_taco
In global workspace theory, there is bottleneck representation After this talk, I strongly believe bottleneck is feature, because bottleneck forces model to learn nice compression way, which we call inductive reasoning
Comment from : @kimchi_taco


@deepbayes6808
Information theory to the rescue of unsupervised learning, who would have thought ;)
Comment from : @deepbayes6808


@joeysipos
So is he saying there is some algorithm to take compression of say Large Language Models and generalize them to work with say vision?
Comment from : @joeysipos


@Morimea
Interesting talk about compression, thanks!
Comment from : @Morimea


@deepbuilds4553
If anyone is wondering, the equations for statistical learning theory for supervised learning were drawn from his phd thesis: wwwcsutorontoca/ilya/pubs/ilya_sutskever_phd_thesispdf
Comment from : @deepbuilds4553


@Achrononmaster
A avatar "@Yuksel Mert Cankus" in the chat nailed it (or at least nailed a good comment on the topic) Because in most practical use cases we can tolerate errors, and because a sentient mind is the final user of a NN, it might not always be so useful to focus on achieving a numerical approximation to Kolmogorov compression There is a profoundly fuzzy goal: find an Ai tool that's useable, better than yesterday's tool, and compute efficientbr (Let's be honest and note the utopian goal of sentient subjective awareness, aka "consciousness" and uploading your mind into silicon is laughable If you end up doing it, you can come back to now in your time machine and tell me all about it)
Comment from : @Achrononmaster


@huyle3597
what's the reference for the inequality he showed at around 4:05?
Comment from : @huyle3597


@EdedML
An interesting exploration of a lot of these points (including Solomonoff induction) is presented in the paper "Playing games with AIs: The limits of GPT-3 and similar Large Language Models" -- highly recommended reading
Comment from : @EdedML


@vev
Scott Aaronson asking question!?
Comment from : @vev


@kenmogibrainworld4844
In the phenomenology of consciousness qualia are great compressors
Comment from : @kenmogibrainworld4844


@IAjayMukhiya
👍
Comment from : @IAjayMukhiya


@jonatan8392
I think your intro about supervised learning is a bit missleading Yes it is true theoretically that test-errortraining-error if the model complexety is small, for instance in terms of VC-dimension brHowever, this VC theory cannot explain the success of deep learning In practice, large models with billions of parameter are used and trained to nearly 0 training-error and STILL the test error is small Applying the VC-dimension argument to these models doesn't give you anything because the observed test-error is magnitudes times smaller than the predicted test-error from the VC-theory
Comment from : @jonatan8392


@tristanwegner
It will be interesting in the future to see how the compression of AI systems gets better and better They have already had neural networks discover physical laws in raw experimental data, but how little data is needed for that? Nobody knows! Maybe it just needs eg a few seconds of airwaves broad spectrum signal This also makes any safety concept for oracle AGIs that involves somehow keeping "safety critical data" from the AGI a very weak approach The world is highly correlated, and a higher intelligence might learn much more from what we give it, than we see in it ourselves
Comment from : @tristanwegner


@nanow1990
Hear me out: unsupervised learning on synthetic data Look up Code-lama-unnatural benchmarks
Comment from : @nanow1990


@odomobo
2 minutes in, and I can already tell this talk is going to be fascinating
Comment from : @odomobo


@NerdFuture
Ray Solomonoff's version of compression is very similar to Kolmogorov's, and Solomonoff's version of induction is predicting the next token Another random Ray thing, in the 1980s he was worried that a smart enough supervised AI would effectively learn the trainer's objective function and turn the tables
Comment from : @NerdFuture


@eskelCz
The compression/prediction could be more simply thought of as pattern recognition
Comment from : @eskelCz


@wrathofgrothendieck
Ilya Sutskver da god
Comment from : @wrathofgrothendieck


@JohnDoe-nv2op
All on top of the "SGD miracle" When we realise that SGD is just crap, all this DL horseshit will fall into oblivion
Comment from : @JohnDoe-nv2op


@specifictoken
내가 가장 좋아하는 대머리!!
Comment from : @specifictoken


@dreamphoenix
Thank you
Comment from : @dreamphoenix


@afrozenator
Talk starts at 0:14
Comment from : @afrozenator


@markpfeffer7487
S tier brain D tier PowerPoint aesthetics
Comment from : @markpfeffer7487


@alonsomartinez9588
Reminds me of VR concerts
Comment from : @alonsomartinez9588


@shinkurt
It makes sense
Comment from : @shinkurt


@simonstrandgaard5503
Excellent talk
Comment from : @simonstrandgaard5503


@binjianxin7830
50:00 the size of the compressor (GPT4) is a salient term in the inequality 😂
Comment from : @binjianxin7830


@hanskraut2018
Ilya is great / sympatic I hope some day you can communicate and exchange some trade secrets so its not too much and not too little Because a 100 secrecy seems counterproductive as well as a 100 openness might lose to people just hiding everything and copycats or even hypothetical worse tactics So much for proprietary information :)
Comment from : @hanskraut2018


@rohanfernando
Supervised learning is essentially an artifice Unsupervised learning is the foundation of Intelligence Kohonen was the true pioneer in this, and everything since seems to be extensions and tweaked variations on his foundational ideas
Comment from : @rohanfernando


@syphiliticpangloss
The guy at the end gets at the point that the semantics are scrambled and these guys are confused about semantics and the expression of the problems Obviously very smart but bad at writing about this stufff/expressing it
Comment from : @syphiliticpangloss


@syphiliticpangloss
Why is he calling "next pixel prediction" and UNSUPERVISED task?
Comment from : @syphiliticpangloss


@GraczPierwszy
well well, brwe'll see what it all comes down to brblah blah blah so far brwe will see the effects in the real world, brnot on the board
Comment from : @GraczPierwszy


@kenwolf887
It's all about entropy and complexity
Comment from : @kenwolf887


@SchoolofAI
What are 100 videos that are MUST-WATCH for AI enthusiasts in 2023?
Comment from : @SchoolofAI


@mamotivated
The eloquence of his delivery was delightful
Comment from : @mamotivated


@JCResDoc94
31:10 GPT models _JC
Comment from : @JCResDoc94


@Alejandro388
he is high on what?
Comment from : @Alejandro388


@Sarmadpervez186
ClosedAI
Comment from : @Sarmadpervez186


@jimlbeaver
We need more people that are at least half as smart as him
Comment from : @jimlbeaver


@jony7779
Does anyone know the name of the theorem being shown at 4:54 so that I can look it up to learn?
Comment from : @jony7779


@lukas-santopuglisi668
thx so much for uploading from Germany!
Comment from : @lukas-santopuglisi668


@WalterSamuels
Why is your company called OpenAI if you're the opposite of open? OpenAI is supposedly the "leader" of the AI space, yet it's the least generous out of all AI companies Meta is constantly releasing new research, tools, open-source software and the rest, while OpenAI is busy playing politics and trying to squash competition It's really quite sad and disappointing No wonder you had such a large exodus of talent Hopefully you'll change your ways
Comment from : @WalterSamuels


@loopuleasa
main openai brainbrthanks for posting this
Comment from : @loopuleasa


@odiseezall
Great beard, keep it
Comment from : @odiseezall


@KemalCetinkaya-i3q
I got it
Comment from : @KemalCetinkaya-i3q


@charlesmarks1394
Great pres Highest tier of insight and scientific communication
Comment from : @charlesmarks1394



Related An Observation on Generalization videos

French Primary Lesson Observation French Primary Lesson Observation
РѕС‚ : La Jolie Ronde Ltd
Download Full Episodes | The Most Watched videos of all time
TEACHING STRATEGIES FOR MOTIVATION (PANGGANYAK) | DEMONSTRATION AND OBSERVATION TEACHING STRATEGIES FOR MOTIVATION (PANGGANYAK) | DEMONSTRATION AND OBSERVATION
РѕС‚ : Binibining Lindsay
Download Full Episodes | The Most Watched videos of all time
5 Best Spotting Scopes of 2025 for Precision Observation and Exploration 5 Best Spotting Scopes of 2025 for Precision Observation and Exploration
РѕС‚ : HowMuchTech
Download Full Episodes | The Most Watched videos of all time
Best Board Games For Kids|Our Favorite Games for 4+ Year Old Kids|Logic Observation u0026 Concentration Best Board Games For Kids|Our Favorite Games for 4+ Year Old Kids|Logic Observation u0026 Concentration
РѕС‚ : TeenyTinySteps
Download Full Episodes | The Most Watched videos of all time
observation of teaching and learning policy and procedure observation of teaching and learning policy and procedure
РѕС‚ : Institute of Commercial Management (ICM)
Download Full Episodes | The Most Watched videos of all time
Observation, Documentation u0026 Reflection of Infants and Toddlers Observation, Documentation u0026 Reflection of Infants and Toddlers
РѕС‚ : Professor LaMarr
Download Full Episodes | The Most Watched videos of all time
Learning From Observation - Artificial Intelligence - Unit-V Learning From Observation - Artificial Intelligence - Unit-V
РѕС‚ : D Sumathi
Download Full Episodes | The Most Watched videos of all time
Learning from observation | AI | Artificial intelligence | Lec-47 | Bhanu Priya Learning from observation | AI | Artificial intelligence | Lec-47 | Bhanu Priya
РѕС‚ : Education 4u
Download Full Episodes | The Most Watched videos of all time
Artificial intelligence for Earth observation Artificial intelligence for Earth observation
РѕС‚ : European Space Agency, ESA
Download Full Episodes | The Most Watched videos of all time
Learning from Observation Part 1 (Artificial Intelligence) Learning from Observation Part 1 (Artificial Intelligence)
РѕС‚ : Md Abdul Masud, PhD
Download Full Episodes | The Most Watched videos of all time


Mission 2 Case Of The Missing Coins | How To Learn Spanish FAST With Netflix (Updated Version) | Project Based Learning PBL Definition, Success U0026 Innovative Learning | How To Make Money With Google Ads (For Beginners) | Best Distance Learning Management Courses In India | Ancient Coins How Were They Made? | Fifa 15 UT How To Make Your First Coins On Fifa 15! | HUGE PELE WAGER! 15 MILLION COINS!!! Fifa 14 Ultimate Team Pinkslips | Young Jeezy U0026 Slick Pulla Performs "Throw This Money" | Importance Of Earning A College Degree | How To Clean Silver Coins At Home || Very Easy And Simple Trick || 100 Working || | "Coin Operated Boy" By The Dresden Dolls Lyrics | Which TYPE OF LEARNER Are YOU?! | Honey And Mumford Learning Styles | Pattern Recognition With Machine Learning | Republic Of Liberia 20 Dollar Coin Silver Content | Canadian Vs American Money | (2013) FIFA 13 Ultimate Team Coins Generator | Dream Meaning Of Money | What Is Coin Collecting Hobby Called | MAKE MONEY With Organo Gold | Investments For Small Amounts Of Money | WWE: "HERE COMES THE MONEY" ► SHANE MCMAHON I THEME SONG | A Super Fun Math Game! The "Trash Can" Game Learning Probability U0026 Place Value | B&b 3 Coins Trevi Fountain Rome | How To Borrow Money From Someone | Fifa 14 | 60 Second Trading Tips | The Rare Gold 550 Coins Method | How To Get Money FAST On Xbox 360 Sims3 | Inquiry Based Learning | Quality Early Learning And Toronto Day Care | Learn And Grow In Preschool Toronto | BUYING/SELLING CHEAP FIFA 13 ULTIMATE TEAM COINS | Fifa 13 Ultimate Team: Selling Coins Cheap | FIFA 13 Ultimate Team Coins For Sale ! | What Are Learning Disabilities? | Why Waste Time Sorting Coins? The Cassida C200 Makes It Easy! | Cassida Corporation