> Brynjolfsson: ”I asked Sundar [about Google losing the initiative]. He didn't really give me a very sharp answer.
> Schmidt: Google decided that work life balance and going home early and working from home was more important than winning. The reason startups work is because the people work like hell. I'm sorry to be so blunt, but the fact of the matter is if you go found a company and compete against the other startups — like [we did] in the early days of Google — you're not going to let people work from home and only come in one day a week.”
It’s pretty cack-handed to publicly talk about your successor like that. While the WFH part of this got a lot of press I wonder if the free-wheeling side-swipe at Sundar Pichai had more to do with the cringe backpedaling.
Also, the humblebrag about his medal!… he’s an investor!… he showed Sam Altman his calculations that OpenAI will need lots of electricity!… he’s an investor!… he wrote a report setting national AI policy that was “only about 752 pages long”!… he’s an investor!
Schmidt has done some amazing things and his achievements will eclipse many others but I do wonder if even he feels a bit of the post-FAANG blues where one misses the glory days of ones peak, over performing and telling everyone about it to show you’ve still got it.
> you're not going to let people work from home and only come in one day a week
Huh, I guess I dreamed the first start-up I worked for (a couple of decades ago) where indeed I only came in one day a week.
Yes of course you "work like hell". We had a nasty leak bug and I set things up so that day or night if the leak was detected my stereo would go maximum volume and play "Straight Outta Compton". How does commuting count as "working like hell" ?
If I'm sat in a car (and once a week I often was) then I'm not working am I? I am useless for several hours each day we do that. Maybe sometimes the CTO (who is in the car, he's driving, he worked from home too) is discussing relevant technology, you know design of our secret sauce schema-less database engine, IP stuff - but then it's also possible we're discussing the album that's playing, or a video game we both enjoyed, or a novel we're both reading.
People who sit in their cars an hour every morning, stressed by traffic and pollution before they even set foot in the office, who then attend back-to-back meaningless meetings and email and slack, and then get back in the goddamn car again for another hour if they're lucky, think everyone who doesn't perform this unhealthy and destructive ritual is a slacker.
This reminds me of the Dave Barry quote: Meetings are an addictive, highly self-indulgent activity that corporations and other large organizations habitually engage in only because they cannot actually masturbate.
One thing I noticed after switching to WFH is that I am no longer dealing with that 15-30 minute "mental haze" that happens right after I finish my 2 hour car commute: That decompression period where my brain is fatigued from being on high alert, white knuckling it down I-580, then I-680, then I-880, watching for hazards, accidents, or just slogging through bumper-to-bumper traffic. I'd walk into the office but my brain was just too fatigued to work without a little time spent just walking around like a normal person.
There are other ways to commute. Commuting via train gave me a chance to go for a walk through parts of the city I'd otherwise not spend a lot of time in. Being stuck in suburban hell currently, working 100% from home is a nightmare scenario for me. Asshole neighbors. Constant noise (far worse than e.g. Oakland). Shit infrastructure (electric, telecom, whatever).
And, in my experience, collaboration almost always suffers. One coworker used to work 100% from his man cave but also refused to invest in getting decent WiFi coverage down there.
then it's also possible we're discussing the album that's playing, or a
video game we both enjoyed, or a novel we're both reading.
So? Even though HN has a hard on for eliminating human interaction, socialization is important.
That said, Schmidt is wrong to pin Google's failures on remote work. Pichai is a fucking moron and Google's toxic culture is destroying both their ability to put out competitive products and to keep anything around long enough to get meaningful market share.
So? Going into the office makes a clear delineation between my personal and professional life. Working from home means I never get around to doing the things I need to do at home. Working from an office means I leave all (well, most) of that shit at the office. IOW, don't shit where you eat.
I commuted every day in London, for 6 months, going to work in Croydon, from Blackfriars. It was the worst six months of my life, you make it sound like you're the only one commuting via train, every morning I would get covered in armpits, train delay, train canceled, people standing on top of each other.
Commutes can be great when they aren't crowded. But overcrowded trains and subways, highway traffic jams, and risk-your-life city cycling are commuting nightmares.
Or cities that have seen a uptick in crime but refuse to either fund security measures or empower the existing workers to do anything about it on mass transit.
Seattle used to have a great transit setup between the busses and the light rail. But post Covid, the security and enforcement was scaled back dramatically, making it a real crap shoot on how safe a ride could be. This is slowly being recognized and changing [1], but I don't need the stress of being stabbed or shot each morning on the way to work.
On other ways to commute: You assume that people who commute by car do so because they want to, instead of going by train. I don't think that is the case.
On people who refused to get a good WFH setup (e.g., WiFi): I bet those are terrible people to work with in the office, too.
>On people who refused to get a good WFH setup (e.g., WiFi): I bet those are terrible people to work with in the office, too.
Huh? That's some crazy generalization Batman. Many people who rent have little control over their home office setups,. especially in a bad housing market.
Really? Pray tell what other methods I would have to commute? The closest bus, train or subway service is about the same distance as driving to work with a car. Parking at work is also free, but I have to pay for parking at the bus stop, or the train stop, or the subway stop. Oh and there's no way to walk there because I would have to walk along the shoulder of a 4-lane highway where the speed limit is 65mph.
There’s no point in arguing with people on HN. The same people downvoting you are the same people who refuse to use AI because it’s “not perfect”. They would rather work from home and be made redundant and unemployable than be productive and valuable.
No, the people arguing with the parent are the ones who have actual life experience of living in places where there's no reasonable access to public transit (read: most of the USA) and who aren't naive to think that everywhere is a perfect little urban world where you have usable choices in your method of transport.
I live inside a major metro area. My closest public transit is a 10 minute drive by car, or a several hour walk by foot, and that's a 3x a day bus. If I wanted to actually access useful public transit, I would need to hike 3+ miles over about 1500' of elevation change, on roads with no sidewalks. And I'm in one of the most public transit friendly metro areas in the US.
No. You make a choice to live far or unconvinced from your place of work. And that is totally fine. But it doesn’t change the fact you are less productive working from home than in the office. Can argue all you want but it doesn’t change the facts.
Haha a few companies making bold claims it worked despite lost productivity and realising later that being in the office was more productive doesn’t align with your “fact”.
You haven't produced an iota of evidence to support your claims as being facts.
It's almost like such decisions aren't made based on a single binary argument, but on a lot of different factors. Sure, I could live closer to work. I'd be paying more for a smaller house, worse schools, more local traffic, more noise, neighbors that are a little closer than comfortable. But yes, then I'd be able to get on the bus and ride it to work instead of using my car. I doubt I'd be more productive though because my sleep quality will suffer, I'll have more stress dealing with things like getting the kids to school, or worrying if my car is going to get broken into or hit-and-run while it's parked on the street. Instead of being able to take a short 10 minute break to read to the kids, or to help cook lunch or dinner, I'll just wander down to the cafeteria and make myself some cheap tea and grab a granola bar. Sure, I don't get to see the kids as much, and my partner can't work full-time hours anymore because I'm not there to help, but at least the bus stop's right there eh?
And perhaps I don't really care to be as productive as possible for your definition of productive. I don't live to work, I work so I can live how I want. Somehow, I've managed to still have a job; In my time I have seen plenty of 'productivity maximalists' come and go.
Lots of great startups have remote teams so this argument by Schmidt is pretty weak. Google is hybrid, not even remote. Also, I don't think that well motivated and engineers work or produce less while WFH. It could be the opposite. Less interruptions are key, as explained in Peopleware. But bad managers love controlling people in open plan offices.
Regardless, I found it surprising Schmidt didn't talk about other stuff that differentiates startups. Smaller teams, a lot less red tape, a lot more ownership, less politics... Google should really follow Steve Jobs 2011 advice to Page about focus. Breaking down into a conglomerate was a great idea to bring focus and agility, but it has not been fully executed and lots of the different resulting entities seem dysfunctional.
Interestingly, re: red tape, Schmidt has been quoted from this same talk as saying if you get caught stealing IP then just “hire a whole bunch of lawyers to go clean the mess up, right?”
>>I don't think that well motivated and engineers work or produce less while WFH.
Thats the whole problem isn't it. Years back I was involved in a project(not WFH), like everyone worked at office. But the manager/lead was sort of a totally disengaged person. He would sit in a conf room all day. And come up with some weird things about freedom to work the way people liked.
There was no ticketing system, the code repo had no real commit and PR rules, no stand ups, no bug tracking, no feature backlog, nobody measuring how the project was going, and if it was even making progress. Only real visible thing about progress was a odd demo every now and then. To make things worse, there were two senior engineers who seem to make their own power structures and bully people into doing whatever they wanted. The project folded quickly enough of course.
Sure if every one was motivated and organised enough, things could have been better. But most people are not. And if people aren't engaged enough they just do whatever they want, or even worse do nothing.
If you are keeping lights on in a project, and have lots of old employees things do work fine with remote work. I think building things quickly, especially big things quickly, does require close 1-1 collaboration, and engagement. I don't think its too much to ask. Sometimes thats just how things work.
I have worked for 2. Granted, they're unlikely to be massive, unicorn successes - but they are successes, none the less.
My opinion is largely the era of the unicorn is essentially gone. Except for extremely rare moonshots (like OpenAI), markets just don't simply have many multi-billions of value for a disruptive startup.
I believe tailscale is remote only. Personally remote doesn’t work for me, I feel my motivation and mental health go into a tailspin after a while, but there is no denying some companies have made it work.
At some point, someone is going to realize that people are not going to slave away for the privilege of living in a rental and drinking meal replacement shakes because they're too busy to eat.
People who worked at Google early on were always going to get rich, people who bust ass for Google now might wake up to a 10,000 man layoff in the morning.
They're having quite a bit of trouble convincing people to work in the office again, nevermind 6 day weeks or 10 hour days.
We raised a generation who learned companies have no loyalty and no consideration for their wellbeing, or the wellbeing of the society they're a part of.
I don't think the next generation is always doomed to repeat the past, otherwise we'd never have invented labor laws. Life would still be as horrible as it was, but seeing as it isn't, I would say that is evidence that it can get better, and I think we're now in the process of learning to make it better.
Eric has a penchant for saying things he thinks before thinking about how they might be heard. He famously said, "There is no privacy, get over it. If you're not doing anything wrong you don't have anything to worry about."
Imagine my complete surprise that the guy in charge of Google when sex with subordinates was rampant, and who gave an exec who was sacked for sexual harassment a ninety million dollar parachute, is upset that there aren't more people in the office.
Google did to AI what Bell did to magnetic recording: retarded it's progress for decades because they were afraid it would harm their main product.
For those that don't know:
>In early 1934, Clarence Hickman, a Bell Labs engineer, had a secret machine, about six feet tall, standing in his office. It was a device without equal in the world, decades ahead of its time. If you called and there was no answer on the phone line to which Hickman’s invention was connected, the machine would beep and a recording device would come on allowing the caller to leave a message.
>Soon after Hickman had demonstrated his invention, AT&T ordered the Labs to cease all research into magnetic storage, and Hickman’s research was suppressed and concealed for more than sixty years, coming to light only when the historian Mark Clark came across Hickman’s laboratory notebook in the Bell archives.
>AT&T firmly believed that the answering machine, and its magnetic tapes, would lead the public to abandon the telephone.
The same thing was true for google, with the very real threat that something like perplexity will eat their lunch by having a monthly payment and no adds. With google relegated to a second tier API endpoint.
20 years of AI advancements were used for better add targeting in gmail, and in the months after chatgpt came out, a better spell checker.
I took this more as Schmidt being disappointed in what Google has become. He really did build Google, its completely valid for him to be mad about the state of the company culture wise.
But he also was the "adult in the room" that basically destroyed whatever moral core Google had as a startup.
It might have the been the correct move from a business standpoint and there was no way they'd keep "don't be evil" as a compass, but Eric Schmidt was there to push it all over the edge. The company's current PR issues[0] is perfectly seeded in his contribution to the company.
[0] PS: not counting the company potentially broken up a few years down the line.
Or maybe mad that he wasted his life having no work-life balance and then realizing that lots of companies with work-life balance are productive and profitable.
I’m ordering something from Europe where the factory is shut down for the entire month of August. I just have to wait an extra month for that item. The company still gets my money because I need the item.
I also use a piece of software written by a European company. I will get no releases/patches this month. But that software company still got my money. I need/want the software.
I wonder how many hustle culture US tech founders find out that a lot of Europeans just don’t work at all for a whole month and finally start questioning some of this bullshit.
What are these companies you speak of? I would like to take them over in a leveraged buyout, chain the employees to their desks, and make an immediate 9% return
You might be surprised that chaining employees to their desks doesn’t make them more productive.
It’s easy to figure out that a lot of companies collect the same checks from their customers regardless of how much their employees burn themselves out. Google doesn’t magically get to charge more money for cloud storage because they decide to work their engineers harder. Labor is only a small component of the cost of that product.
Sorry, but that smells like bullshit. No European company I worked at or know of entirely shuts down in August. That's just a meme. There's still people at work keeping things going that month. Vacation approvals need to be requested in advance and staggered because of that to keep things going while people are on vacation.
And if you're still giving them money despite being absent for a month because like you say you need their product, that's more like survivorship bias where that company probably has a monopoly and no competition so you're forced to only buy from them but that's no your average European company, not by a long shot.
The company I work in now, the managers check in on things even on vacations.
> The company I work in now, the managers check in on things even on vacations.
Bruh, my manager would never do that to me. It’s against our team norms. If your manager makes you do a significant amount of work when they “check in” or requires you to answer their messages, that policy is illegal and you’re owed your time off back. You need a new job or a new manager.
> Vacation approvals need to be requested in advance and staggered because of that to keep things going while people are on vacation.
You’re just saying what I said: that the companies in Europe make extended time off possible by staggering leaves. Some companies stagger leave, others decide to shut down or slow down operations. I never said that every company in Europe shuts down to make holidays work, but I can guarantee you that every company in countries like Denmark, Finland, and France are allowing their employees to take 25 days of leave (plus national holidays) because that’s the law.
The important part is that the individual gets to take time off. Whether the entire company shuts down or not is up to the specific business. The two companies I mentioned decided that it’s easier for them to shut down for a month because their work is essentially in a queue. For the factory making physical goods, they can receive orders while they are closed and catch up on them when they are opened again. For the software company, they can continue selling their software digitally while the company is on leave and their users just wait an extra month for software updates.
This never-ending work culture in the USA isn’t even helping overall production and productivity all that much.
Not working for a whole month just sounds awful. I have a lot of hobbies outside of work, but it is still the most interesting and fulfilling part of my life. Even though I need breaks for it now and then doesn’t mean I won’t miss it.
I get to spend time with wonderful people, solving interesting puzzles, building something that people love and that makes their days better. I used to work in game dev, so it brought people joy, but now I make developer tools, and it using better tools makes them love their jobs more, and makes them more effective. Some of the people who use the tools are literally doing cancer research or other very meaningful stuff.
How could I get the same feeling of fulfilment from the most amazing vacation?
And what happens to this paradigm if you don’t really love your job more than your family and friends and just work out of necessity to survive?
Only half of all Americans are “very satisfied” with their jobs.
Lower income people are less satisfied with their jobs than higher income people.
What about people who work jobs that are physically demanding where time off is literal physical rest?
This idea of work as fulfillment is highly classist IMO.
And anyway, you wouldn’t like to have a month every year to spend on your own personal coding projects that don’t become owned property of your employer? Nobody said you have to spend your time off sipping margaritas on the beach.
Really fascinating to see such a mix of blind spots, insecurity and, yet, good insights on full display here. It’s almost as if he’s a normal human being at the end of the day, despite his circumstances.
It’s almost as if what he says carries more weight due to his status and he should be more careful in what he says publicly, especially if it’s charged. He deserves the flogging he’s getting.
The biggest difference between him and a "normal smart person" is they realize they have blind spots and choose not to publicly spout opinions on them.
This thing just gets repeatedly uploaded, I guess deletion is not that useful now.
Put that aside, Eric Schmidt did told the truth (maybe except AI and NVidia stock thing) and offered the public a glace of how things actually work within the industry.
One important thing he mentioned was (after simplified) "just steal, and deal with justice later", which actually mirrors what Steve Jobs once said "Great artist steal". This should remind everyone, that if you want to have significant achievement in this industry, you at least needs to guard against thefts, because there WILL BE (maybe ALREDY are) people who comeout with a "be shameless or be dead" mindset after been educated by people like that man.
(Also, I found it quite funny that one of Eric's girl friend has probably took his lesson too)
From what I understand, it’s over the controversy in his statement about Google losing competitive advantage because they chose work life balance over in-office collaboration.
Genuine question: what would be the refutation of this opinion, if any?
Out of options like "Google are not actually losing", "Google are losing but not because they chose work life balance", "Google are, indeed, but so be it", etc.
I'm not a googler, but my take would be that Google was a smaller ship with technical founders at the helm who understood the problem and acted accordingly with first mover benefits. I remember when I started to use search engines in grade school (AskJeeves, AltaVista, etc) and Google just had the most relevant answers.
Google is now a behemoth that seems to just chuck money and promo packets at problems (Stadia, Google Cloud, Nest, etc) with no clear vision for them. It kind of reminds me of the examples in this blog: https://gist.github.com/chitchcock/1281611
There's also self selection going on - people joining startups generally have more vested interest in solving a problem and will work longer and harder, too. So I think there's some kernel of truth what Eric Schmidt was saying. How much enthusiasm can a person have about increasing the bottom line of a mega public company?
I don’t think startup aficionados and hustle culture members realize that the world runs on C-players.
You’re right, startup people have a drive to build great things and they burn the midnight oil and they get divorced during the development of the original iPhone, etc.
But the people who keep the lights on at most companies especially big ones like Google are a sea of average, steady, stable employees who do their tasks and not much else, go home at 5 or maybe earlier and have no interest in burning themselves out so that the CEO can have a second private jet.
And you don’t want to run your company on all A-players and 10x developers because once the thing is built they’ll get bored and move on to the next thing, or they’ll hit a mental breakdown and rage quit because they’re the bulb burning brightest.
Those C-players will just keep clocking in, they won’t be poached by more attractive places to work, they won’t be difficult to cover for when they go on leave, and they won’t write code that goes over the head of other C-players.
Technically, a developer at a public company should work harder to boost the stock price because they’re getting shares as compensation. But at the same time, making 5% more in stock value isn’t worth working 50% harder.
I personally don’t think there’s anything wrong with the culture at Google. People who wish Google would be exciting again like Schmidt are just grasping at nostalgia. Boring business may be boring but boring business is excellent business.
> But the people who keep the lights on at most companies especially big ones like Google are a sea of average, steady, stable employees who do their tasks and not much else, go home at 5 or maybe earlier and have no interest in burning themselves out so that the CEO can have a second private jet.
This was my moment of clarity during the pandemic. I was working insane hours and it just occurred to me then - why am I working so hard and sacrificing my health? So what if I don't work another 10 hours on top of the 90 I was already working? Is my CEO working this hard (numbers-wise)?
For me - I knew then I was content with being a C-level (probably even a D) player.
Your CEO may very well be working those hours too except for good reason since he's value aligned+ highly leveraged and able to capture the benefits from the extra work
CEOs also just don't have the kind of work most of us do. They have tons of latitude to direct their day. They can skip meetings. They have executive assistants that they can offload much of their admin work. If they find themselves too in the weeds they can hire a COO. Being able to be self-directed changes how you perceive "work".
Exactly. From director or VP level to C-suite are people who can hire and delegate. If you’re an IC and you are drowning in work you have to live with it.
Yes at the extremes where you have sufficient leverage the hours are irrelevant, but for any pre-IPO stage CEO my statement is clearly not a misconception.
If you want a "diverse" team, having both kinds of above stated people on the team is crucial. The above mentioned A players can't do the steady boring work the so called C players can do & likewise that the C players can't do what the A players can. Also C players can be great at questioning all the bleeding edge new tech A players want to implement. For reasons like this, I think it's important that more senior positions are made up of both kinds of people.
Good managers will help everyone identify which kind of person they are & help them have work that fits them best. They'll keep the C players motivated & the A players feeling challenged.
Google always advertised their workplaces as fun. They pioneered the table tennis at work meme.
So it’s odd for them to blame work life balance, when they have always pushed their work place as more fun, colorful and relaxing than other work places.
Google is no longer a startup. Google is competing for a significant number of talented people. The startup mentality will not easily work for decades of competition with the other wealthiest tech companies.
Small company gets big, gets taken over by MBAs who all hire McKinsey to help themselves take over more of the company. The people at the top become professional ladder climbers, eventually too much of the leadership got their for the wrong reasons and the company is unable to insulate themselves from corporate's bad decisions.
Kinda like Goodheart's law but for promotion criterion.
The number of fairly senior people I've encountered who spend their whole career presenting things that have never been built as if they exist is exhausting.
I feel like that was not that big of a deal. I mean was it a bad thing to say for multiple reasons? sure. But its still a good hour of content and no one says perfect things all the time.
it was too truthful from what I can ascertain. Always interesting when you get to hear what someone actually thinks. Most tech talk/interviews are access / pr fluff.
- Alphabet have made a switch from a (mostly) tech-driven company to a pure finance-driven one, when this happen you can expect a bit of growth, even a sharp one for a little time, than an inexorable decline;
- the more and more centralized web means search engine became dysfunctional beasts try to pour water to specific mill instead of being open search and for a company living on search more than anything else it's a problem, of course there is GMail a project born when Google was tech-driven, but again GMail is search, so Drive, and in that "specific ___domain" they have substantial competitors and they can't have innovation being financially-driven now;
- LLM-push a way to reach a kind of dummy semantic holy grail of research while very popular is a substantial failure. Yes it have a certain wow-effect, but results are such low quality that the wow effect will not last longer and current "better-than-Alphabet" competitors in that field still have to see any meaningful and durable growth and profit.
Long story short the anti-remote-working is just another droplet in a substantially lost PR/élite battle to force people in the city, the sole way they have to remain alive because finance and services can only live at large city scale where they own anything and they rent/sell services to anyone. In a spread world they simply die killed by SME innovation and personal ownership that value substantial innovation against services.
IMVHO the big-tech model, a cleptocracy born out of Xerox tech once they have found a way to make it anti-user, is at it's end. I do not know what could happen next because so far ALL élites want such model to rule slaves, and we have lost much of intelligence, competence of the Xerox time, and no one else in the world seem to be there, China included, who was able to surge as an industrial power but still lag behind in software, even in the current sorry state of IT. But it's clear that the service model of big tech is dead. Or they found a way to reinvent or they are done.
It will be interesting to see if Schmidt’s comments about the future of AI being mostly a China-US game will come true, or if less powerful models will ultimately be more useful than single centralized powerful ones.
The analogy here might be between nuclear weapons and drones; the former are controlled entirely by a small number of countries, but the latter probably have more of a direct role to play in the future of warfare - and yet aren’t costly or difficult to make at all. The assumption tends to be that the most powerful well-capitalized tech always wins, but I don’t think that is necessarily the case.
This answer was responding to a question about smaller countries competing against China, the US, and larger countries in AI. “The rich” here refers to the richer countries, not people in general.
Please read/listen to the link before commenting on out of context sections.
Again, the question was about competition in the ecosystem, not some kind of class warfare thing. He was describing the reality of it, not making a justification. He then went on to describe how other places aren’t allocating the resources that China and the Us are.
The way you framed it, which implies that you didn’t actually watch the video, is that Schmidt was making a comment justifying the rich dominating the poor. It was no such thing.
it's funny how both threads here on hn, the top comments focus on his quote badmouthing work life balance.
when anyone with half a brain can see that work life balance has much less impact on a tech company than replacing the tech founders with the likes of him and a sea of business major middle Managers...
Eric ran Google while the Android team often worked 7 days a week in office (famously: “bacon Sundays”) and still managed to blow every advantage they had over iOS by making repeatedly terrible strategic decisions.
But yeah, he’d have won AI by keeping butts in seats…
Listened to the first few mins. Did he say anything controversial in the? The part about Google work-life balance isn't that big a deal. It's no secret that startups grind more.
This. I worked hard at Google for years because I was pationate about my projects and my colleagues. Then over the past few years especially starting around dragonfly and maven we started getting more of an us vs them culture pushed down from the top. Layoffs were the end for me. I didn't exactly quiet quit but I toned it down and only worked on what I absolutely felt like.
Then jumped over to a startup where we work hard but in a much more organic and empowering way. People come in when it makes sense.
To me startups just inherently have a bigger carrot at the end of the stick. being an early engineer in a successful startup can bring life-changing money. In a 1T company with 100,000 of engineers it just isn't possible to have that incentive. your best work is often a drop in the ocean that some VP will shelve at the fist sign of trouble. Yeah working weekends can get you that promotion quicker and give a salary bump, but it's not the same as watching the company IPO and becoming an engineer overnight.
Funny that pretty much describes the company I work for. People would work their asses off. Pull all-nighters, work through holidays, etc. Then the rounds of layoffs started happening and now nobody cares at all about the company. Everyone just does the minimum while planning their next thing.
"The country is going to have to learn critical thinking. That may be an impossible challenge for the US." (ts 30:57)
"Impossible" is a bold qualifier. Maybe he was exaggerating - maybe not. Will critical thinking necessarily become more difficult - to learn, teach, exercise - with AI? It's possible it will help people become better thinkers, but I don't think that's a guarantee.
What is Google supposed to win at? Being the most popular search engine? Being the highest grossing ad agency? Having the highest average salaries? Selling the most used cars? Curing cancer? Owning more generators than anyone else? Contributing money to youth groups? How do we judge whether Google is winning?
At about 39 minutes in, he's asked if efforts analogous to seti at home can be used to get around the scaling problems with training the next round of big models.
He gives a strongly NVidia oriented answer that I happen to think is dead wrong. Pushing more and more GPU/Memory bandwidth into more and more expensive packages that are obsolete after a year or two isn't the approach that I think will win in the end.
I think systems which eliminate the memory/compute distinction completely, like FPGA but more optimized for throughput, instead of latency, are the way to go.
Imagine if you had a network of machines, that could each handle one layer of an LLM with no memory transfers, your bottleneck would be just getting the data between layers. GPT 4, for example, is likely a 8 separate columns of 120 layers of of 1024^2 parameter matrix multiplies. Assuming infinitely fast compute, you still have to transfer at least 2KB of parameters between layers for every token. Assuming PCI Express 7, at about 200 Gigabytes/second, that's about 100,000,000 tokens/second across all of the computing fabric.
Flowing 13 trillion tokens through that would take 36 hours/epoch.
Doing all of that in one place is impressive. But if you can farm it out, and have a bunch of CPUs and network connections, you're transferring 4k each way for each token from each workstation. It wouldn't be unreasonable to aggregate all of those flows across the internet without the need for anything super fancy. Even if it took a month/epoch, it could keep going for a very long time.
I think you need higher algorithmic intensity. Gradient descent is best for monolithic GPUs. There could be other possibilities for layer-distributed training.
That's exactly it though, these CEO's are so divorced from reality that they forget. Schmidt knows how silicon valley really operates. Because he's probably been involved in a lot of this shitty corporate behavior.
I have a question out of curiosity: Has anyone or anything ever succeeded in deleting/removing anything from the Internet once it goes up?
I’ve realized that anything that goes online will be (perhaps unintentionally) leaked, hacked, uncovered, archived, altered, restored, referenced, and what not.
