DeepSeek

FAQ

Explainer by Waqas Ahmed:

* Q: What is DeepSeek and why is it causing a stock meltdown?

A: The Chinese company DeepSeek has released an AI model that is as good as any of its American counterparts and has made it open source. This has fundamentally changed the economics and politics of the rapidly emerging AI industry, which has so far been led by an oligopoly of American tech companies trying to position Large Language Models (LLMs) as the defining technological breakthrough of this century, and themselves as the custodians of its secret sauce.

There’s a lot of talk about DeepSeek costing only about $6 million to build, although that figure does not include research and development. And despite export controls, DeepSeek did manage to exploit a non-trivial number of the high-tech chips we were trying to keep from them. Yet it’s still a massive shock to the U.S. industry.

* Q: What are LLMs and how did they take off?

A: A 2017 paper titled “Attention is all you need” was a turning point in the AI industry. The paper described a method of creating a machine learning model that could produce human-like text with unprecedented accuracy and scale using an architecture called “transformers.” These “transformers” considerably improved a class of models called Large Language Models (LLMs). LLMs use massive amounts of text—books, articles, emails, recipes, faqs, everything—to create internal mathematical representations of relationships between billions of words and phrases—or more accurately, between combinations of tokens that are found in a natural human language.

Before 2017, LLMs were not very useful, but “transformers” changed that. By processing large amounts of text using the transformer architecture, these models could now “learn” what words mean in different contexts, and detect nuances computers had never been able to before, allowing these models to output extremely relevant text in response to a user prompt or question.

* Q: How did the AI hype start?

A: OpenAI became the first American company to demonstrate that if you take a snapshot of the whole known internet and all digitized books in existence without worrying too much about copyright law, you can create a model so good that its output would be almost indistinguishable from that of a DC bureaucrat with mediocre intelligence. However, OpenAI showed, its model could be trained to have expertise in different domains and could give in-depth answers to very specific questions. Its model passed coding exams, the bar exam, and graduated business school. The results were so shocking that OpenAI went out and claimed it was worth a gazillion dollars and that the future of humanity depended on it.

* Q: What is the current state of the AI industry?

A: OpenAI, partially owned by Microsoft, was the first to release a major LLM product, ChatGPT in November 2022. Soon after, Meta released its own model, LLaMa, and Google released Gemini. All three companies had massive amounts of text to train their models on, but an LLM needs another crucial ingredient: computing power to process that text and then to generate responses to user queries. The leading company that makes the computing machines is Nvidia, whose stocks grew exponentially as response when OpenAI/Microsoft, Google, and Meta led LLM wars ensued.

The computing machines are called GPUs—Graphic Processing Units. They had originally been invented to process computer graphics for games, such as 3D rendering. Later they became popular because their parallel processing capabilities made them ideally suited for cryptocurrency mining. Now, it turns out, they are also great at AI data processing for similar reasons. Nvidia has basically been riding waves of booms as different markets discover new uses for its product.

Over the past few years, Meta, Google, Microsoft, and OpenAI have managed to hoard hundreds of thousands of the most advanced GPUs, and get preferential treatment from both Nvidia and its supplier, the world primary manufacturer of semiconductors, TSMC.

The American tech industry has been taking significant steps to align itself around AI. Companies have been acquiring startups, recruiting top AI researchers, and pouring resources into developing their proprietary primary AI models (called foundational models), creating a flow of investment into AI and related technologies, such as cloud computing, advanced chip manufacturing, and data infrastructure. This is all a bid to secure dominance in what they claim is the next frontier of technological innovation.

* Q: How is China involved?

A: As a part of its larger effort to contain China, the U.S. government has been on a mission of stopping Chinese companies from becoming leaders in different areas of technology. It has done so by wielding control over global supply chains and protecting American tech companies from competition in the process. The U.S. blocked Huawei’s entry into the United States just as it was overtaking Apple to become the second biggest smartphone manufacturer in the world; it stopped European countries from installing Huawei manufactured 5G infrastructure when it was clearly more economical; and most recently, it passed legislation banning TikTok, a Chinese social media app that had become massively popular in United States and whose recommendation algorithm no American social media app had been able to outperform.

The U.S. claim that Huawei and other Chinese tech companies are inextricably linked to China’s geopolitical strategy and put Western companies and people at heightened risk of surveillance and corporate espionage is, of course, grounded in reality. DeepSeek isn’t shy about how much data it collects on its platform, including even your keystrokes:

We collect certain device and network connection information when you access the Service. This information includes your device model, operating system, keystroke patterns or rhythms, IP address, and system language. We also collect service-related, diagnostic, and performance information, including crash reports and performance logs. We automatically assign you a device ID and user ID. Where you log-in from multiple devices, we use information such as your device ID and user ID to identify your activity across devices to give you a seamless log-in experience and for security purposes.

However, because DeepSeek is open source and can run locally on a separate device, Chairman Xi Jinping’s prying eyes can be shielded.

Maintaining global technological dominance is one of the key concerns U.S. policymakers have repeatedly cited, and have identified AI as a crucial technology in maintaining that dominance. In 2018, when the U.S. government was in the process of banning Huawei, it realized that it would need to do the same with downstream technologies like semiconductor chips, the main component used in CPUs and GPUs. The severe chip shortage due to global supply chain disruptions during Covid-19 showed that advanced chips are a global supply chain bottleneck and a scarce resource. By 2022 the Biden administration had put comprehensive sanctions on China, stopping the export of these chips to the country and preventing Chinese AI companies from accessing the latest and most efficient GPUs. At the same time, it passed the CHIPS act, subsidizing national semiconductor manufacturing with over $50 billion.

* Q: Why is everyone suddenly so into AI?

A: The over-the-top marketing and snake oil salesman level of pushing by the U.S. AI industry has caused somewhat of a freakout among less technically literate government policymakers. Many industry insiders claimed that advances in LLMs could soon lead to creation of Artificial General Intelligence (AGI), basically a computer that thinks like a human being and is good at many different tasks. Some have already sounded the alarm that it can become evil and self-aware. But even its detractors have agreed LLMs are a game-changing technology that will fundamentally change how we interact with computers."

(https://www.dropsitenews.com/p/deepseek-openai-lina-khan-sam-altman)