2024/08/19 11:08
Former Google CEO Eric · Schmidt's latest discussion on the rise of AI, global competition and technological evolution
2024/08/19 11:08
Text: Web3 Sky City · City Lord
Schmidt highlighted the complex factors behind technological advances, such as increased computing power, the development of new algorithms, and the market's insatiable quest for intelligent systems. He also discussed the potential impact of AI on the labor market, data privacy, antitrust, and national security, and made recommendations on how to maintain a competitive edge in this rapidly changing environment.
The city lord especially pointed out that as Schmidt, who has served the Ministry of Defense for a long time, his remarks and positions need to be identified, and I believe that readers will be able to understand them for themselves.
Short-term breakthroughs and far-reaching impacts of artificial intelligence:
Schmidt predicts that in the next one to two years, AI will usher in important breakthroughs, especially in the areas of context window expansion, AI agents, and text-to-action. These technological advancements will enable AI systems to handle complex tasks more efficiently, beyond current limitations. This advancement will not be confined to the field of technology, but will profoundly affect all levels of society, including education, healthcare, government, and business. He emphasized that the potentially transformative impact of the development of these technologies could be even more profound than the rise of social media on society.
The U.S.-China Game in Global Technology Competition:
Schmidt gave a detailed analysis of the fierce competition between United States and China in the field of artificial intelligence. He noted that United States currently leads in technology, talent and resources, but sustaining this advantage will require sustained high investment and international cooperation, especially with allies such as Canada, to ensure a sustainable supply of energy and resources. He emphasized that the future of AI is not just a technology race, but also a strategic game between countries, involving national security, economic competitiveness, and global leadership. Schmidt warned that United States needs to invest more to deal with China's rapid rise in AI and maintain its global dominance in this area.
Monopoly and Innovation Challenges of Tech Giants:
Discussing the current dominance of tech giants, Schmidt pointed out that the monopoly of companies such as NVIDIA in the field of AI is due to their strong technical capabilities and capital advantages. He mentioned that while there are competitors in the market, challenging the position of these tech giants requires huge investments and technological innovations. He also expressed concern about how these giants will continue to drive technological innovation in the future, arguing that capital-intensive AI development could lead to a fundamental change in the software development model, moving from open source to closed source, further cementing the monopoly position of the giants.
The impact of AI on society and the labor market:
Schmidt explores the potential impact of AI on society, the economy, and the labor market. He believes that while AI technology may replace some repetitive tasks, it will also increase the importance of highly skilled jobs and drive productivity in complex tasks. He also expressed concern about the social inequalities that AI could bring, noting that rich countries would benefit more from AI, while poorer countries could be left behind. In addition, Schmidt called for tighter regulation of AI to address issues such as data privacy, intellectual property, and the spread of misinformation.
Adversarial AI and Security Challenges:
Schmidt singled out the potential threat of adversarial AI, predicting that there could be specially designed AI systems to attack and disrupt other AI systems in the future. This development will bring new challenges to safety and ethics. Schmidt suggested that the tech community and governments need to work together to study how to prevent these risks and develop regulations and technical standards to ensure the safety and trustworthiness of AI. He also mentioned that research in this field will become an important direction for the development of science and technology in the future, and may receive more attention in universities and research institutions.
=Web3 Sky City Full Text Collated Edition=
Chairing Professors:
Today's guests need no introduction. I first met Eric about 25 years ago when he came to Stanford Graduate School of Business as CEO of Novell. Since then, he has held key positions at Google, starting in 2001, and joined Schmidt Futures in 2017. In addition, he has been involved in many other projects, which you can check out. So Eric, if I can, I'll start there.
First of all, what do you think is the direction of AI in the short term? I guess you define that as the next one to two years.
Schmidt:
Things are changing so fast that I feel like every six months I need to give a new speech about what's coming. There is a group of computer science students here, can anyone explain to the rest of the class what a "million token context window" is? Please say your name and tell us what it does.
Student: Basically, it allows you to prompt with a million marks or a million words.
Schmidt:
So you can ask a million-word question. Anthropic is 200,000 tokens, to 1 million, and so on. As you can imagine, OpenAI has a similar goal.
Can anyone here give a technical definition of an AI agent? Again, computer science.
Student: An AI agent might be something that acts in some way. It could be calling something on the web to find information on your behalf. It could be many different things along this line of thought. There are all sorts of things going on in one process.
Schmidt: So an agent is something that performs some kind of task. Another definition is that it is an LLM, state, and memory. Again, computer scientists, can any of you define "text to action"?
Student: Instead of converting text into more text, let the AI trigger an action based on this.
Schmidt:
Another definition is the Python language. I never wanted to see a programming language survive. Everything in AI is done in Python. A new language called Mojo has just emerged, and it looks like they've finally solved the AI programming problem, but we'll see if it can really survive Python's dominance.
There is also a technical issue. Why is NVIDIA worth $2 trillion while other companies are struggling?
Student: The technical answer is that most of the code needs to run with CUDA optimizations that are currently only supported by NVIDIA GPUs, so other companies can do whatever they want, but unless they have 10 years of software experience, you won't have machine learning optimizations.
Schmidt:
I like to think of CUDA as a C programming language for GPUs, and I like the idea that I like it. CUDA was founded in 2008. Even though I've always thought it was a terrible language, it took hold. There's also a notable insight: there is a set of open-source libraries that are highly optimized for CUDA and none of the others do so. All the people who build these tech stacks completely ignore this in the discussion. These libraries are technically known as VLLMs, and there are many similar libraries that are also highly optimized for CUDA, which makes it difficult for competitors to replicate.
So, what does all this mean? Next year, you'll see very large contextual windows, proxies, and text-to-action apps. When these technologies are delivered at scale, they will have a huge impact on the world, far beyond the impact of social media. Here's why: In the context window, you can use it as a short-term memory, and I'm struck by the length of the context window. The technical reasons are related to the difficulty of the service and calculations. The interesting thing about short-term memory is that when you type in information and ask questions, like reading 20 books and using the text of the books as a query, and then asking what they are about, it forgets the middle part, which is similar to how the human brain works.
Regarding agents, there are people who are now building LLM agents by reading and understanding knowledge in areas like chemistry, then testing it, and then adding it back to their understanding. It's very powerful. The third aspect is text-to-action. I'll give you an example: let's say the government is trying to ban TikTok. If TikTok is banned, I suggest you say the following to your LLM: copy TikTok for me, put my preferences in it, make this program and publish it in the next 30 seconds, and then within an hour, if it doesn't catch on, do something like that. That's the order. You can see how powerful this is.
If you can translate to any number command from any language, that's essentially Python in this scenario. Imagine that everyone on the planet has their own programmers who actually do what they want to do instead of those who don't work as required. The programmers here know what I'm talking about. Imagine a programmer who isn't arrogant actually does what you want to do without you having to pay a high price. And the supply of these programmers is unlimited.
Professor: All this is going to happen in the next year or two?
Schmidt:
Soon. The above three things, I am sure, will happen at the same time in the next wave. So you ask what else will happen. I fluctuate every six months, so we're in an odd-even oscillation state. At the moment, the gap between the cutting-edge models (now there are only three) and the others seems to be widening. Six months ago, I was convinced that the gap was narrowing, so I invested a lot of money in some small companies. However, now I'm not so sure about it.
I'm having conversations with some big companies and they've told me that they need $10 billion, $20 billion, $50 billion, or even $100 billion. The Stargate project requires $100 billion, which is very difficult. My good friend Sam Altman thinks this could probably cost about $300 billion, if not more. I pointed out to him that I had calculated the amount of energy required.
To be fully public, I went to the White House on Friday and told them we needed to be best friends with Canada. Because the people of Canada are really good, they helped invent artificial intelligence, and they have a lot of hydroelectric resources. Because we as a country don't have enough strength to accomplish this. Another option would be for Arab countries to fund the project. I personally like Arabs, I spend a lot of time there, but they may not abide by our national security rules. And Canada and United States are among the Big Three that we can all agree on.
As a result, in these $100 billion to $300 billion data centers, electricity is starting to become a scarce resource.
By the way, if this reasoning is followed, you may ask me why I am discussing CUDA and NVIDIA? If all $300 billion goes to NVIDIA, you know what to do in the stock market. This is not a stock recommendation, though, and I'm not a licensor.
Professor:
Part of the reason is that we will need more chips, but Intel gets a lot of money from the United States government. AMD is trying to build a fab in Korea.
Schmidt:
If you have Intel chips in any of your computing devices, raise your hand. The monopoly ends there.
Professor:
But that's the point. They did have a monopoly once, and now NVIDIA has a monopoly. So are these barriers to entry?
Speaking of CUDA, are there any other options? I spoke to Percy Lange the other day. He's switching between TPU and NVIDIA chips, depending on what he has access to. This is because he has no choice.
Schmidt:
If he had unlimited funds, today he would have opted for NVIDIA's B200 architecture because it is faster. I'm not suggesting – it's good to have competition.
I've had a long talk with AMD's Lisa Su. They've built something that can convert the CUDA architecture to their own architecture, called Rokam. It's not quite functional yet, but they're working on it.
Professor:
You've been working at Google for a long time, and they invented the Transformer architecture. Thank you to the amazing people over there, like Peter, Jeff Dean, and everybody. At the moment, OpenAI seems to have lost the initiative. In the latest rankings I've seen, Anthropic's Claude is at the top of the list. I had asked Sundar but he didn't give me a very clear answer. Maybe you have a more pointy or objective explanation of the situation there.
Schmidt:
I'm no longer a Google employee. Google is more focused on getting employees home early and working from home in terms of work-life balance, rather than chasing for winning. In contrast, startups are successful because their employees work hard. While this may seem blunt, the truth is, if you come out of college and start a company and want to compete with other startups, you can't just have employees come one day a week.
PROFESSOR: In the early days of Google, so did Microsoft.
Schmidt:
Now it seems that in our industry, for too long, companies have won in a really creative way and dominated a certain area rather than making the next transformation. This is well documented. I think founders are special and they need to be in control, although it can be difficult to work with them because they put a lot of pressure on their employees. As much as we don't like Elon's personal behavior, look at what he's getting from his staff. I had dinner with him, and he was flying. I was in Montana, and he was flying to a midnight meeting with x.ai at 10 p.m. that night. Think about it.
Different places have different cultures. I was very impressed with TSMC. They had a rule that just graduated PhD students, i.e. good physicists, were to work in the basement of the factory. Can you imagine having a physics Ph.D. in the United States do this? Unlikely. It's a different work ethic.
My strict demands on my work are because these systems have network effects, so time is of the essence. Whereas, in most businesses, time is not that important, you have a lot of time. Coca-Cola and Pepsi will still be there, the competition between them will continue, and everything will be cold. When I deal with telcos, a typical deal takes 18 months to sign. There is no reason to spend 18 months doing anything, it should be done as soon as possible. We're in a period of growth and maximizing earnings, but that also requires crazy thinking.
For example, when Microsoft made a deal with OpenAI, I thought it was the stupidest idea I've ever heard. Outsource your AI leadership to OpenAI and Sam and his team? This is crazy. At Microsoft or anywhere else, no one is going to do that. Today, however, they are becoming the most valuable company, certainly going toe-to-toe with Apple. Apple doesn't have a good AI solution, but it looks like they made it work.
Student:
How will AI play a role in terms of national security or geopolitical interests, especially in competition with China?
Schmidt:
As the chair of an AI committee, I delved into this. We wrote a report of about 752 pages, which summarizes as follows: We are currently leading the way and need to maintain this advantage, which requires a lot of financial support. Our main clients are the Senate and the House of Representatives, which has also led to the introduction of the CHIPS Act and other similar policies.
If cutting-edge models and some open-source models continue to evolve, there may be only a handful of companies that will be able to compete in this space. Which countries have such capacity? These countries need to be well-funded, talented, have strong education systems, and have the will to win. United States and China are two of the main countries. As for whether other countries will be able to participate in this, I'm not sure. But what is certain is that in the future, the competition between United States and China for intellectual supremacy will be a major struggle.
The United States government has essentially banned the export of NVIDIA chips to China, though they are reluctant to admit it publicly. We have about 10 years of technical superiority in sub-DUV chips, i.e. sub-5nm chips. This advantage puts us a few years ahead of China, much to the displeasure of China. This policy was enacted by the Trump administration and supported by the Biden administration.
PROFESSOR: Is Congress listening to your advice and making large-scale investments, obviously the CHIPS Act is an example.
Schmidt:
In addition, we need to build a huge AI system. I lead an informal, temporary, non-legal group that includes some of the most common industry insiders. Last year, those members made the case for being the Biden administration's AI bill, the longest presidential directive in history.
We have discussed a central question: how do you detect danger in a system that has learned but you don't know what to ask? In other words, the system may have learned something bad, but you don't know how to ask it. For example, it may have learned how to mix chemicals in some new way, but you don't know how to ask it. To solve this problem, we suggested in a memo to the government to set a threshold, which we call 10 to the 26th power, which is a technical calculation metric. Beyond this threshold, businesses must report their activities to the government. In order to ensure that they are different, the European Union has set the 25th power of 10 to 10. But these values are close enough. I think all these distinctions will disappear because the existing technology will be obsolete. The technical term is called joint training, which basically refers to the fact that individual pieces can be combined together. As a result, we may not be able to protect people from these new things.
Professor:
Rumor has it that OpenAI has to train this way, in part because of power issues. There is no place where they do.
Let's talk about a real war that is taking place. I know that you are actively involved in the war in Ukraine, especially about the "White Stork" program and your goal of destroying a $5 million tank with a $500 drone. How does this change the war?
Schmidt:
I worked for the Secretary of Defense for seven years trying to change the way we run the military. I'm not particularly into the military, but it's very expensive, and I wanted to see if I could help. In my opinion, I basically failed. They gave me a medal, so they might give it to a loser or something. But my self-criticism is that nothing really has changed, and the United States system does not lead to real innovation.
So I decided to start a company with your friend Sebastian · Trun, who was a faculty member here, and a whole bunch of Stanforders. The idea is basically to do two things: to use artificial intelligence in a sophisticated and powerful way in these wars, which are essentially robot warfare; The second thing is to reduce the cost of robots. Now you might be wondering, why would a good liberal like me do this? The answer is that the whole theory of the army is tanks, artillery and mortars, and we can destroy them. We can make the punishment of invading a country (at least by land) basically impossible. It should eliminate that kind of land warfare.
Professor:
This is a very interesting question. Does it give defense an edge over offense? Can you tell the difference?
Schmidt:
Because I've been doing this for the last year, I've learned so much about war that I don't really want to know. One thing you need to know about war is that offense always has an advantage because you can always overwhelm the defenses. Therefore, from the point of view of defense strategy, you better have a very powerful offensive to use when needed. The system that I'm building and others is going to do that. Because of the way the system works, I am now a licensed arms dealer. So I'm a computer scientist, a businessman, an arms dealer. I say with regret. Is this an improvement? I do not know. I don't recommend that you engage in this in your career. I will stick to artificial intelligence. Because of the way the law works, we do this privately, and with the support of the government, it's all legal. It went directly into Ukraine, and then they started the war. Without going into all the details, the situation is pretty bad. I think that in May or June, if the Russians build as expected, Ukraine will lose a large chunk of territory in the process of losing the entire country. The situation is quite bad. If you know Marjorie Taylor Greene, I suggest you remove her from your contact list. Because she's the one who stopped billions of dollars from going to the rescue of an important democracy.
Professor:
Next, I'd like to discuss a somewhat philosophical question. Last year, you co-authored an essay with Henry · Kissinger and Dan ·Hertenloch on the nature of knowledge and how it develops. I also discussed this the other night. For most of history, humanity's understanding of the universe was mysterious, until the advent of the Scientific Revolution and the Enlightenment. In your article, you mentioned that models have now become so complex and incomprehensible that we don't really know what's going on inside them. I quote Richard · Feynman: "What I can't create, I don't understand." "I saw this sentence the other day. But now people are creating what they can create without really understanding the inner workings of it. Has the nature of knowledge changed in some way? Do we have to start accepting only the surface of these models that they can't explain to us?
Schmidt:
I would like to give an example of a teenager. If you have a teenager, you know they're human, but you can't quite figure out what they're thinking. However, we have managed to adapt socially to the presence of adolescents, who will eventually get out of this state. This is a serious question. As a result, we may have knowledge systems that cannot be fully described, but we understand their boundaries and the limits of what we can do, and this is probably the best we can get. Do you think we'll be aware of these limitations? If we can do that, that's great.
The consensus that my group meets weekly is that eventually there will be so-called adversarial AI, where there will actually be companies that hire you and pay to disrupt your AI system. It's like the red team. Unlike today's human red teams, you'll have an entire company and an entire AI systems industry whose job it is to disrupt existing AI systems and find their loopholes, especially those that we can't figure out. It makes a lot of sense to me. It's also a great program for Stanford. If you have a graduate student who has to figure out how to attack one of these large models and understand what it does, then this is going to be an important skill for building the next generation. Therefore, it makes sense to combine the two.
Professor:
Now, let's answer some of the students' questions. There is a classmate in the back, please say your name.
Student:
You mentioned before, and it's related to the comments now, let the AI actually do what you want. You just mentioned adversarial AI, and I wonder if you can elaborate on that in more detail. It would seem that in addition to the obvious increase in computing power, you can get higher performance models, but the question of making them do what you want to do, seems to be partially unanswered.
Schmidt:
Well, you have to assume that the current hallucination problem will decrease, with the advancement of technology, and so on. I'm not saying it's going to go away. And then you also have to assume that there are efficacy tests, so there has to be a way to know if this thing is successful. In the case of TikTok competitors I mentioned, I'm not suggesting illegally stealing someone else's music. What would you do if you were an entrepreneur in Silicon Valley? I hope you're all Silicon Valley entrepreneurs. If your product is successful, you'll hire a large group of lawyers to handle the follow-up issues. But if no one uses your product, then it doesn't matter even if you steal everything. Of course, don't quote me.
Silicon Valley usually conducts these tests and deals with follow-up issues. This is common practice. I think you're going to see more and more performance systems, and even better testing, and ultimately adversarial testing, which will keep it within a framework. This term is called chain-of-thought reasoning. It is believed that in the next few years, you will be able to generate a thousand steps of chain-of-thought reasoning, just like making a recipe. You can run it, actually test if it produces the right results, and that's how the system works.
Student:
Overall, you are very optimistic about the potential for AI advancements. I'm curious, what's driving this progress? Is it more computing power? Is it more data? Is it a fundamental or actual shift?
Schmidt:
The answer is all of the above. The amount of money invested is incredible. I basically invested everything because I didn't know who was going to win and the amount of money I followed was so big. Part of the reason is that the early money has already been made, and those who don't know much about it must have an AI component. Now everything is an AI investment, and they can't tell the difference.
I define AI as a learning system, a system that really learns. I think this is one of them. The second point is that there are some very complex new algorithms out there, and they're kind of post-Transformer. I have a friend and long-time collaborator who invented a new non-Transformer architecture. A group I funded in Paris claimed to have done the same thing. There are a lot of inventions out there, and there's a lot of research at Stanford University. Finally, the market believes that there are unlimited returns for the invention of intelligence. Let's say you put $50 billion into a company, and you have to make a lot of money from intelligence to pay it back. We may experience some huge investment bubbles, and then it will resolve itself. This has always been the case, and it may be so now.
PROFESSOR: You mentioned earlier that leaders are distancing themselves from other people.
Schmidt:
Now there is a company in France called Mistral, and they are doing very well. I'm obviously an investor. They have already made a second version, and their third mode is most likely closed because it is too expensive. They need income and can't offer their model for free. The debate in our industry about open source versus closed source is fierce. My entire career has been built on people's willingness to share software in an open source way. Everything I do is based on open source. Much of Google's foundation is also built on open source. The work I do is mainly in the technical field. However, the huge capital cost could fundamentally change the way software is built.
My opinion of software programmers is that they will at least double their productivity. There are currently three or four software companies trying to make that happen, and I've invested in all of them during that time. They are all working to increase the productivity of their software programmers. I recently came across a very interesting company called Augment. I often think of a programmer who says it's not our goal. We're targeting software programming teams of 100 people with millions of lines of code, and no one knows what's going on. This is a very good application of artificial intelligence. Will they make money? I hope so, but there are a lot of problems here.
Student:
At the beginning, you mentioned that the combination of contextual window extensions, proxies, and text-to-actions would have unimaginable impact. First of all, why is this combination important? Second, I know that you are not a prophet and cannot predict the future, but why do you think it is beyond our imagination?
Schmidt:
I think it's mostly because the context window allows you to solve the recency problem. Current models take about 18 months to train, including six months of preparation, six months of training, and six months of fine-tuning, so they are always outdated. And through the context window, you can type in the latest happenings and ask questions about the Hamas-Israel war in context, which is very powerful and makes it as up-to-date as Google.
In the case of proxies, I can give an example. I started a foundation to fund a non-profit organization. I don't know much about chemistry, but there's a tool called ChatCrow, which is a system based on large language models that can learn chemistry. They run the system to generate chemical hypotheses about proteins, which are then tested in the lab overnight and the system learns. This is a huge facilitator for fields such as chemistry and materials science. This is a proxy model.
I think that as long as there are a lot of cheap programmers, the concept of text to action can be understood. I don't think we understand what happens when everyone has their own programmer. This is also your area of expertise. I'm not talking about simple tasks like turning lights on and off. I imagine another example, let's say you don't like Google, you can say, build a Google competitor for me. Yes, you can do that personally. Build a Google competitor for me, search the web, build the user interface, make a good copy, and add generative AI in interesting ways. Finish it in 30 seconds and see if it works. A lot of people believe that incumbents, including Google, are vulnerable to this kind of attack.
Professor:
Now, let's take a look. Slido sends a number of questions, some of which have already been uploaded. Last year, we discussed how to stop AI from influencing public opinion and spreading misinformation, especially during the upcoming election period.
Schmidt:
We need to think about short-term and long-term solutions. In the upcoming global elections, most of the misinformation will be on social media, and the current organizational capacity of social media companies is not sufficient to effectively regulate this information. TikTok, for example, has been accused of favoring a certain kind of false information, although I have no proof. I think we're in a mess.
The nation needs to learn critical thinking, which can be a daunting challenge for United States. Just because someone tells you something doesn't mean it's true.
Professor:
Will we go so far that no one will believe in something real? Some call it an epistemological crisis. Now, Elon · Musk says he never did something, but how to prove it?
Schmidt:
We can illustrate with the example of Donald · Trump. I think there is a trust problem in our society, and democracy can fail because of it. The biggest threat to democracy is misinformation, because we have become very good at it.
When I was managing YouTube, the biggest problem we had was people uploading fake videos, resulting in people dying. We have a no-death policy, and trying to solve this problem is shocking and scary. This was before the advent of generative AI.
Professor:
I don't have a good answer, but there is one technical issue that seems to alleviate the situation, and that is public key authentication. When Joe · Biden speaks, why not use SSL-like digital signatures? Can a celebrity, public figure, or someone else have a public key?
Schmidt:
It's a form of public key that provides some sort of certainty, like when I send a credit card to Amazon, I know it's Amazon.
I had a paper with Jonathan Haidt on which it didn't make an impact. He's a very good communicator and I probably wasn't. My conclusion is that the system is not organized as we say it is. CEOs are usually maximizing revenue, and to do that they want to maximize engagement, and the way to maximize engagement is to maximize anger. The algorithm chooses anger because it generates more revenue, so people tend to support something extreme. This is a problem in every way and must be addressed.
In democracies, my solution to TikTok is based on something we've discussed privately before. When I was a kid, there was something called the equal time rule. TikTok isn't actually a social media platform and more of a form of television. Every TikTok user in United States uses the app for an average of 90 minutes a day and produces 200 videos, which is a lot of usage. While the government does not implement equal time rules, it may be a direction worth considering, requiring some form of balance.
Student:
The first is about the economic impact of the labor market. The impact has been slower than initially expected, especially in terms of the labor market. There are also questions about the customer service staff. Do you think academia should receive AI subsidies, or should it work with large companies?
Schmidt:
I've been pushing the university to build a data center. If I were a faculty member in the Department of Computer Science, I would be frustrated that I couldn't work with graduate students to develop algorithms for doctoral research because I was forced to work with companies. And these companies are not generous enough in this regard. A lot of faculty members spend a lot of time waiting for Google Cloud credits, and it's a bad situation. We want universities in the United States to succeed in this, so I think it's the right thing to do for them to get those credits.
Regarding the impact of the labor market, I listen to the opinions of real experts. As an amateur economist, I believe that university education and high-skilled tasks will have good prospects because people will use these systems. I don't think these systems are fundamentally different from previous waves of technology. Dangerous jobs and jobs that do not require human judgment will be replaced.
Student:
What about the transition from text to action and its impact on computer science education? I believe that computer science education should adapt to the changing times.
Schmidt:
I'm assuming that a computer scientist in an undergraduate will always have a programmer buddy. When you learn your first for loop, there will be a tool that becomes your natural companion. The professor will explain the concepts, and you'll be involved in that way.
Student: Regarding the discussion of non-Transformer architectures, I think the state model is a direction that is being discussed, but now the focus is more on context.
Schmidt:
I don't have a deep enough understanding of mathematics, but I'm glad that this creates employment opportunities for mathematicians because math is very complex here. Basically, these are different ways to do gradient descent and matrix multiplication, with the goal being faster and better. As you know, Transformers is a systematic method of multiplying at the same time. Here are my thoughts. It's similar to that, but the math is different. Let's see.
Student:
In your essay on national security, you mentioned the situation in China, United States, and other countries today. The ten countries down from the next cluster will either be United States allies or have the potential to become United States allies. I'm curious what you think of these ten countries. They are a bit of a middleman, not formal allies. How likely are they to join the work that keeps us safe? What's stopping them from joining?
Schmidt:
The most interesting country is India, because the top AI talent comes from India and comes to the United States. We should let India retain some of its top talent, not all, but some. And they don't have the extensive training facilities and programs that we have here. In my opinion, India is a swinging country in this regard. Japan and Korea are clearly in our camp. Taiwan software is terrible, so this won't work. The hardware is great. And in the rest of the world, there aren't many other good options. Europe is screwed up because of Brussels, and this is nothing new. I spent 10 years fighting them. I'm working very hard to get them to amend the EU Act. They still have various restrictions that make it very difficult for us to conduct research in Europe. My France friend spent all his time fighting Brussels. As my personal friend, Macron is working this. So I think France has a chance. I don't think Germany will come, and other countries are not strong enough.
Student:
I know you're a trained engineer and I think you're called a compiler. Given the capabilities you envision these models have, should we still take the time to learn to code?
Schmidt:
Yes, because at the end of the day, it's a cliché, why learn English if you can speak it? You'll learn better. You do need to understand how these systems work, and I know that.
Student:
I'm curious if you've explored a distributed setup. I'm asking this question because, of course, it's difficult to make a large cluster, but MacBooks are powerful. There are a lot of small machines around the world. So, do you think the idea of folding at home or something like that would apply to training these systems?
Schmidt:
Yes, we have looked at this issue very carefully. So the way the algorithm works is that you have a very large matrix, and you basically have a multiplication function. So think of it as going back and forth. And these systems are completely limited by the speed of memory to CPU or GPU. In fact, next-generation NVIDIA chips have combined all of these capabilities into a single chip. The chips are now so big that they all stick together. In fact, the package is so sensitive that both the package and the chip itself are assembled in a clean room. So the answer looks like supercomputers and the speed of light, especially the memory interconnect, really prevail. I don't think splitting large language models (LLMs) is possible at the moment.
Professor:
Jeff Dean mentioned in a talk last year that it is possible to divide a model into different parts, train them separately, and then combine them.
Schmidt:
But to do that, you need to have tens of millions of these models, and the speed of asking questions becomes very slow. He mentioned that it would take eight, 10 or 12 supercomputers to make it happen, but it wasn't at his level.
Student:
On the subject of data privacy, I learned that the New York Times had sued OpenAI for using their work for training.
Schmidt:
I think there could be a lot of similar lawsuits in the future, and eventually some kind of agreement will be reached, like stipulating that a certain percentage of revenue is required to be paid for the use of certain works, like ASCAP and EMI in the music industry. This model may seem a bit outdated, but I think it will eventually work that way.
Student:
In the AI space, there seem to be some companies dominating the market and overlapping with the larger companies that are the focus of antitrust regulations.
Schmidt:
I've worked on Microsoft's split case in my career, but it didn't work out in the end; I've also tried to keep Google from being split, but again, it didn't work. Therefore, I think the trend is not split. As long as these companies do not become monopolies like John · Rockefeller, the government is unlikely to act.
These big companies dominate because only they have the capital to build data centers. I have friends Reid and Mustafa who made the decision to split the business to Microsoft because they couldn't raise tens of billions of dollars. As for the exact numbers, you may need to let Reed tell you.
Student:
Finally, I would like to know what impact these developments will have on countries that are not involved in the development and calculation of cutting-edge models.
Schmidt:
Rich countries will get richer, while poor countries will have to do their best. This is actually a game of rich countries, which requires huge capital, technical talent, and strong government support. Globally, many countries face a wide variety of problems, especially when resources are scarce. They need to find partners to work with to solve these problems.
Professor:
I remember the last time we met, you were at the AGI House for a hackathon. I know you spend a lot of time helping young people create wealth and are passionate about it. Do you have any advice for those who are writing a business plan for a course or writing a policy proposal or research proposal for their career?
Schmidt:
I teach a course in business school, so you should come and listen to it. I'm amazed at how quickly you guys come up with new ideas.
In one of the hackathons I attended, the winning team was tasked with getting a drone to fly between two towers. They were in a virtual drone space, using Python to generate code, and successfully completed the mission in the simulator. It can take a week or two for a good professional programmer to do this. I think the ability to prototype quickly is very important because part of the problem with entrepreneurs is speed. If you can't prototype with these tools in a day, you'll need to rethink because that's exactly what your competitors are doing.
So, my biggest piece of advice is that it's okay to write a business plan when you're starting to think about starting a company. In fact, you can have the computer write for you, as long as it's legal. It's important to use these tools to prototype your idea as soon as possible, because there may be people doing the same thing at other companies, universities, or places you haven't been to.
PROFESSOR: Thank you very much.
This article is from Xinzhi self-media and does not represent the views and positions of Business Xinzhi.If there is any suspicion of infringement, please contact the administrator of the Business News Platform.Contact: system@shangyexinzhi.com