on March 18, the dark side of the moon announced that Kimi, its intelligent assistant, had a context length of 2 million words.

October 2023, the dark side of the month released Kimi, an intelligent assistant that supports the context length of 200000 Chinese characters, which was the longest of the large model services that can be used in the global market at that time. Now Kimi's ability limit has been increased ten times, reaching the international leading level. At present, the model with the longest context in the world is Gemini 1.5 Pro, which was launched by Google in February 2023. Under the most extreme circumstances, it can reach 10 million token, but it is not open to use. The daily open to the outside world is only 1 million token length.

context technology is one of the core capabilities of large language models, which determines the depth and breadth of the model's understanding of information. Supporting a longer context length enables the model to maintain high accuracy when processing a large amount of information, and further broadens the application scenarios of the model, such as the analysis and understanding of the complete code base, the agent that independently completes multi-step complex tasks, the lifelong assistant that does not forget key information, and the multi-modal model with a truly unified architecture. Just like a computer's RAM, the operating system retains the real-time context of all applications, and because of the sufficient context length, LLM can retain a large amount of user context like a "reasoning computer.

large model context length expansion of the existence of the "impossible triangle": text length, attention and computing power.

Transformer architecture relies on self-attention mechanism to process the input sequence. As the sequence length increases, the computational complexity of the self-attention layer increases quadratically, where n is the sequence length. This means that for longer sequences, the model requires a large number of calculations, which also puts forward higher requirements on the calculation force, which limits the infinite expansion of the context length of the large model from two aspects. The longer the context text, the more difficult it is for the model to focus full attention and fully understand the user's intent; with attention constraints, short text cannot fully interpret complex information; and processing long text requires a lot of computing power, which increases costs. According to the current input and output price of GPT-4 Turbo API, if you want to use a full 128k length for input and output, the price of a question and answer exceeds that of 30 yuan.

With the deepening of large model long text technology, the dawn of AI application landing. At present, the industry has gradually formed a consensus, even a large model of hundreds of billions of parameters can not completely avoid the illusion and nonsense. Compared with short texts, long texts can assist the model in judging semantics by providing more context and detail information, further reducing ambiguity, and more accurate induction and reasoning based on the provided facts. Long text technology can not only solve some problems that were criticized in the early days of the birth of large models, enhance some functions, but also a key technology to further promote the industry and application landing. With Kimi taking the lead in breaking through long text technology in China, AI-related applications are also expected to accelerate further. At present, the dark side of the moon is aimed at the 2C track, and the long text ability in the 2B track will have more room to display in the future.

The demand for computing power is also expected to rise. Taking into account the architectural factors of the Transformer itself, the increase in context length will inevitably lead to an increase in computing power consumption. Even in the case of continuous optimization in the industry, the demand for computing power will still increase to a large extent. With the gradual extension of Kimi-driven long text technology, we believe that the major model manufacturers will gradually start the training and research of long text models, and the demand for computing power will be further increased.

We believe that the continued expansion of the length of the large model context is expected to accelerate the landing of AI-related applications, while the demand for computing power will also increase significantly.

AI application: it is recommended to pay attention to Jinshan office (688111, overweight), new software (688590, unrated), hkust xunfei (002230, buy), flush (300033, unrated), caixun shares (300634, unrated), Shanghai steel union (300226, overweight) and other companies

AI computing power: it is recommended to pay attention to zhongke shuguang (603019, buy), Haiguang Information (688041, Buy), Cambrian-U(688256, Unrated), Yunsai Zhilian (600602, Unrated), Runze Technology (300442, Unrated), Huatie Emergency (603300, Buy) and other companies

Other tools: Star Ring Technology-U(688031, unrated)

risk tips

technology landing less than expected; policy regulatory risk