From left: Satoshi Matsuoka (Director, RIKEN Center for Computational Science) / Fumiyoshi Shoji (Director, Operations and Computer Technologies Division)
Introduction
The RIKEN Center for Computational Science (“R-CCS”) continues to take on the resolution of social challenges using the supercomputer Fugaku※1, which boasts world-leading performance. Jointly developed by RIKEN and Fujitsu, Fugaku shot to prominence through its droplet-dispersion simulations during the COVID-19 pandemic. The visualization research conducted during the spread of infection made a strong impression on the public about how science and technology can contribute directly to social challenges. In recent years, beyond conventional simulation fields such as weather forecasting, materials development, and astrophysics, new R&D has advanced through fusion with generative AI (large language models: LLMs※2).
In this article, through interviews with R-CCS Director Satoshi Matsuoka and Fumiyoshi Shoji, who leads Fugaku’s operations and computer technologies division, we explore how Fugaku has evolved into a supercomputer that is “directly useful to society,” its response to the global generative AI boom, and its expectations for domestic LLMs. As R-CCS advances a plan to transition first-line inquiry handling on the Fugaku support site to the generative AI chat “AskDona,” why adopt generative AI using RAG※3 (Retrieval-Augmented Generation) technology? We unpack the results and challenges, and even the vision leading to a future “AI copilot.”
(Interview supervision / GFLOPS Co., Ltd. — Morimoto & Suzuki | Interview text / Google “Gemini 2.0” and OpenAI “GPT o1 pro mode” | Photos / Karina Tanida @instagram)
Topics
- Social transformation brought by Fugaku’s world-leading performance — how supercomputers impact real-world fields such as medicine and disaster prevention
- The power of supercomputers behind the scenes of LLM development — why supercomputers are indispensable for large language models with vast numbers of parameters
- The significance and strategy of developing domestic LLMs — what benefits AI optimized for the Japanese language and cultural context brings
- R-CCS’s practical approach to the hallucination problem — how to make use of AI that is useful even without being 100% accurate
- The front line of Fugaku support-site operations and the role of the generative AI chat “AskDona” — why even simulation beginners can make use of advanced computing resources
- The future opened by supercomputers × generative AI: a new current of social return and industrial revitalization — the path toward new innovation and the possibilities beyond
Social transformation brought by Fugaku’s world-leading performance
─ What triggered Fugaku’s sudden rise to public awareness, and what technical breakthroughs or decisions lay behind its development?
Director Matsuoka (hereafter, Matsuoka): Fugaku became widely recognized through its simulations of droplet and aerosol dispersion during the COVID-19 pandemic. TV, newspapers, and web media widely reported Fugaku’s visualization research on the spread of infection, and its name recognition surged across Japan all at once. Supercomputers had been well known among researchers and specialists, but during the pandemic Fugaku became something that drew attention from the general public as well.
Around 2020, Fugaku was in the very stage of being assembled toward completion. At that moment COVID-19 became a global crisis, and infection countermeasures became an urgent task in Japan too. So we repurposed the advanced fluid-computation technology we had cultivated in internal-combustion-engine and aerodynamic simulations for the analysis of droplet dispersion. Being able to visualize, on a scientific basis, the finding that “wearing masks and ventilation are effective” and present it to the public was hugely significant.
On November 19, 2024, Fugaku achieved its 10th consecutive world No. 1 in “HPCG※4 (High Performance Conjugate Gradient),” an international supercomputer ranking that competes on the processing speed of the conjugate gradient method actually used in industrial applications. It continues to maintain world-leading performance, and there is great significance in applying that computing power to solving social challenges such as medicine, industry, and disaster prevention. Supercomputers had originally been used in diverse simulation fields such as weather forecasting, materials development, and astrophysics, but Fugaku clearly demonstrated that potential in a new social need — infection countermeasures. High-precision simulation contributing directly to dispelling social anxiety, and presenting the importance of masks and ventilation on a scientific basis, was a textbook example of how a supercomputer can contribute to society.
Data source: Google Trends (https://www.google.com/trends)
The power of supercomputers behind the scenes of LLM development
─ In recent years, generative AI (LLMs) has sparked a worldwide boom. How do supercomputers support this rapid development?
Matsuoka: AI has seen several booms throughout history, and this is positioned as the so-called “fourth AI boom.” In the past, neural networks were in the spotlight, followed by a number of technical milestones such as CNN※5 (convolutional neural networks), GAN※6 (generative adversarial networks), and RNN※7 (recurrent neural networks). Then, with the advent of the Transformer, LLMs (large language models) blossomed, bringing the current generative AI boom.
In fact, the relationship between supercomputers and AI had been deepening since the early 2010s. Around 2012, Google Brain※8 training a neural network using computing resources on the scale of 1,000 CPUs is one example, but after that GPUs became widespread and the processing speed of deep learning improved dramatically, making supercomputer-class computing environments indispensable for AI research. LLMs in particular handle vast numbers of parameters and volumes of data, so training requires enormous computing resources. Building a model on the scale of a trillion parameters can require occupying a supercomputer for months and investments on the order of tens of billions of yen — it has expanded into a domain that cannot exist without supercomputers.
Such LLMs are expected to have extremely large value for business and social implementation. Traditionally, supercomputers contributed to solving scientific and industrial problems through advanced simulation and generated enormous economic effects. But LLMs can be applied to an even broader range of uses and are easy to turn into services and products, so they can attract investment and development funding and have the potential to explosively expand economic value. We have incorporated AI technology into research and practice from an early stage. Building on its success in simulation, Fugaku too is working to broaden into diverse AI applications, including generative AI. In future supercomputer development, we believe an era is coming in which fusion with AI is essential — not merely a simulation-only machine, but keeping diverse AI tasks, especially LLMs, in mind, and actively incorporating GPU-like※9 accelerators and dedicated hardware.
The significance and strategy of developing domestic LLMs
─ Momentum is building in Japan to grow domestic LLMs. Amid fierce performance competition with overseas LLMs, where is the significance in Japan deliberately advancing its own development?
Matsuoka: There are several clear reasons for developing domestic LLMs. First, an LLM is not merely a language-processing technology; the cultural and social background and values reflected in its training data have a large influence. The grammar, honorific expressions, and subtle nuances peculiar to Japanese can be difficult to reproduce sufficiently with models developed overseas. A domestic LLM can build a model optimized for the Japanese language, culture, and social context, enabling more natural and accurate communication.
Ensuring technical independence and competitiveness is also an important point. LLMs form the core of modern AI technology and will play an increasingly foundational role across a broad range of fields. If we depend entirely on other countries for this core technology, there is a risk of facing some constraint in the future or losing competitiveness. Retaining the ability to develop LLMs domestically is equivalent to maintaining the power to create and improve state-of-the-art models ourselves. This can be likened to defense capability — the strategic importance lies in having the technical strength to build it when needed.
Furthermore, applications to fields of science and technology, such as AI for Science※10 that we focus on, are a major motivation for developing domestic LLMs. LLMs have wide-ranging applications and the potential to create new value across broad domains such as science, medicine, education, and manufacturing. In highly specialized fields in particular, models need to learn specialized knowledge that general LLMs cannot fully cover. A domestic LLM can incorporate the specialized data and knowledge held by Japan’s research institutions and companies, powerfully advancing scientific and technological progress.
Of course, developing a high-quality domestic LLM requires large-scale training datasets, enormous computing resources leveraging supercomputers, and collaboration with experts across many fields such as linguistics, information science, and computational science. We will make the fullest use of computing resources such as Fugaku and advance optimization for the Japanese language and specialized domains. In the future, domestic LLMs can be expected to be used in many areas such as medicine and education, enriching people’s lives. For that, environmental development is essential not only in R&D but also in diffusion. Like a notary that can guarantee trust and quality, we want to smoothly deliver technology to society and cultivate the soil in which domestic LLMs are widely accepted.
R-CCS’s practical approach to the hallucination problem
─ Many companies are concerned about hallucination (AI generating information that does not exist) and information-leakage risks. How does R-CCS think these concerns can be eased to advance the use of generative AI in practice?
Director Shoji (hereafter, Shoji): Indeed, if many companies expect “100% accurate answers,” the hallucination problem will be a major barrier. But we believe we need the mindset of not necessarily seeking 100%, and instead “offloading most of the enormous searching, summarizing, and analysis work that humans do.” If it becomes 95% correct, human manual work is greatly reduced, and that alone is sufficient value. Furthermore, using technology that combines reference information, such as RAG, can reduce errors. Introducing generative AI to the Fugaku support site is one such example.
I also feel it is our responsibility for a research institution like RIKEN to serve as a model case and, by openly sharing successful cases and lessons learned from failures, create an environment where domestic companies can adopt these technologies with confidence. Generative AI has only recently appeared, and it is natural that companies still lack technical understanding. That is precisely why we take the lead and make its effectiveness and challenges visible. Doing so should spread a shared understanding that “it doesn’t become as big a problem as feared” and “hallucinations do occur, but at this level it’s well within tolerance.”
The front line of Fugaku support-site operations and the role of the generative AI chat “AskDona”
─ Why did you decide to introduce generative AI to the support site of Fugaku, which has world-leading computing power? Please tell us the aims and background.
Shoji: Fugaku is a world-class supercomputer, but its usage and optimization methods are extremely varied. The doors are wide open — not only private companies and researchers but even high-school students and individuals can use it after the necessary application and review — yet actually mastering it required understanding vast manuals and FAQs and finding the right settings and commands. This high barrier was a major obstacle for new users.
However, generative AI like ChatGPT appeared, and if you ask in natural language, an answer comes back immediately. I felt this intuitive interface was “definitely usable.” If we build a mechanism that answers while referencing internal documents using RAG technology, users should be able to get accurate guidance simply by asking, “I want to do X — how?” RAG (Retrieval-Augmented Generation) also has the aspect of having the LLM reference relevant source documents and data in real time for the answers it generates, curbing hallucination. In other words, rather than merely asking a large language model, it generates answers using Fugaku’s specialized manuals and technical guides as “backing,” so more accurate and context-aligned responses can be expected.
This lets users instantly perform information searches that used to take hours, greatly shortening the time to problem resolution. Naturally, hallucinations cannot be made zero at this point. But backing them with RAG greatly reduces them, and in operation for Fugaku, in addition to answers from AskDona※11, we also include links to primary sources, further suppressing the impact of hallucination. Even in actual prototype operation, cases where users obtain answers more smoothly than expected are increasing. By improving the model over time and running a feedback loop, accuracy and reliability should improve further.
Domains with large volumes of difficult documents, complex regulations, and technical specifications that take time for humans to search and understand are especially advantageous. When handling highly specialized and complex information such as Fugaku’s usage guides and system specifications, or scientific and technical literature, generative AI’s ability to summarize and extract is extremely effective. Even within companies there are mountains of hard-to-process information — for example, complex machinery manuals in manufacturing, vast bodies of security-related standards documents, and patent summaries. Applying generative AI × RAG there should enable efficiency that manual effort could never keep up with.
The future opened by supercomputers × generative AI: a new current of social return and industrial revitalization
─ Beyond support improvements using RAG technology, what kind of future are you looking toward? Please share your outlook.
Shoji: One ideal is for a user to simply tell the AI a rough idea like “I want to run this kind of simulation” or “I want to try these parameters,” and the AI automatically performs the necessary computational procedures, resource settings, and even the assembly of the actual simulation commands. Normally, operating and optimizing a supercomputer requires advanced expertise, but if such a mechanism is in place, advanced computation that used to require specialized skills should be opened up to a much broader range of people. For example, submitting compute jobs and retrieving results, loading the right modules and libraries, reserving with the job scheduler, securing disk capacity — work that users previously had to set up manually after understanding the system configuration — could be automated by AI, including optimization. This would greatly reduce time, effort, and the technical learning cost, letting anyone quickly make use of advanced computing resources.
Introducing generative AI to the Fugaku support site is not only about the immediate effect of improving the convenience of supercomputer operation. It is also a “model case” of using cutting-edge technology to improve the research and development environment. The barrier to using the supercomputer lowers for users, freeing up their time for new challenges. As a result, new discoveries and technologies are born and returned to society. In this flow, it becomes easier for other fields and industries in Japan to judge that “if RIKEN is doing it, safety and usefulness are guaranteed to some degree.” This can be a catalyst that spreads a proactive attitude toward AI use across society as a whole.
We are still only halfway there. But through repeated phased introduction and improvement, a more refined support system and computing environment should take shape. RIKEN always values scientific evidence and demonstration. Through efficiency gains and practical examples with RAG, the release of metrics, and transparency in the improvement process, we want to give the use of AI in Japan a firm footing. As society as a whole becomes positive about AI, new value will be created.
─ Finally, what future vision do you have for the increasingly important relationship between generative AI and supercomputers?
Matsuoka: Today’s Fugaku is designed for general-purpose high-performance computing and is suited to a wide range of uses. But with the rapid progress of AI technology, future supercomputers will be required to handle more diverse and advanced computing needs. For example, designs that can fine-tune LLMs even more efficiently, or specialized configurations for generative AI to run smoothly, are conceivable new directions. Our mission is to aim for system designs that match the needs of the time and to realize the computing environments users want. By leveraging the technical knowledge and operational know-how cultivated in the simulation field to develop new computing environments, we hope to accelerate the use of AI for science, industry, and solving social challenges.
As a national research institution, RIKEN not only leads cutting-edge science and technology but also bears the role of returning its results to society. The role Fugaku played during the pandemic showed the power of directly connecting science and society. It demonstrated the potential for information based on scientific evidence to move society. Now we want to reproduce that experience in the field of AI use. If we can apply generative AI technology in practice and show its safety, effectiveness, and efficiency through real examples, it will become an opportunity for many domestic companies to take the plunge into AI adoption. Going forward, we plan to further expand our partners and advance data sharing and infrastructure development. By building a comprehensive research support system, companies and researchers in Japan will be able to use generative AI more smoothly. We believe this will accelerate the creation of innovation and, as a result, raise Japan’s overall technological strength and industrial competitiveness.
Speaker profiles
Satoshi Matsuoka
Ph.D. in Science (1993), Department of Information Science, Graduate School of Science, the University of Tokyo. Professor at the Global Scientific Information and Computing Center, Tokyo Institute of Technology (now Institute of Science Tokyo) from 2001. Head of the AIST–Tokyo Tech RWBC-OIL lab in 2017. In his current position since 2018. Concurrently a Specially Appointed Professor at the School of Computing, Institute of Science Tokyo. Specializes in high-performance computer systems. He led R&D of the TSUBAME supercomputer series, achieving world top rankings on numerous metrics including power efficiency, and engaged in basic research on parallel algorithms and programming for massively parallel computers, fault tolerance, power efficiency, and fusion with big data and AI. ACM Fellow (2009), ACM Gordon Bell Prize (2011), Commendation by the Minister of Education for Science and Technology (2013), and in 2014 became the first Japanese recipient of the IEEE Sidney Fernbach Award, the top award in supercomputing. HPDC Career Award (2018), Asia HPC Leadership Award at SCAsia 2019, and a second ACM Gordon Bell Prize (2021). In 2022, received the IPSJ Distinguished Service Award, the NEC C&C Foundation C&C Prize, and the Seymour Cray Computer Engineering Award (the top achievement award in supercomputing) — the first person in history to win both the Fernbach and Cray awards. Also received the Medal with Purple Ribbon for his long contributions to computer science research. In 2024, selected as one of “HPCwire 35 Legends” by HPCwire (US). IPSJ Fellow.
Fumiyoshi Shoji
Completed the requirements of the doctoral program at the Graduate School of Natural Science and Technology, Kanazawa University (1998); Ph.D. in Science (2000). Research associate at the Information Processing Center (now the Information Media Center), Hiroshima University, in 1998. Joined RIKEN’s Next-Generation Supercomputer Development Implementation Headquarters in 2006, the Operations and Computer Technologies Division of the RIKEN Advanced Institute for Computational Science (now R-CCS) in 2010, and has served as Director of that division since 2014. Engaged in improving the operational efficiency and usability of large-scale HPC systems. ACM Gordon Bell Prize (2010), IEICE Achievement Award (2012), RIKEN Baiho Award (2024).
About GFLOPS Co., Ltd.
GFLOPS provides AI solutions that support corporate efficiency and innovation, with strengths in cutting-edge AI technology and data analysis. In particular, its proprietary solutions combining large language models (LLMs) and RAG (Retrieval-Augmented Generation) technology achieve high answer accuracy and flexibility, and adoption is spreading among many companies.
- Company: GFLOPS Co., Ltd.
- Head office: Shibuya-ku, Tokyo
- Business: Development and provision of AI services leveraging large language models (LLMs) and generative AI technology
※1 Supercomputer Fugaku: The successor to the supercomputer K. Aimed at contributing to Japan’s growth by solving social and scientific challenges in the 2020s and producing world-leading results, it began shared use in March 2021 as a world-class supercomputer in the overall strength of power efficiency, computing performance, user convenience and ease of use, groundbreaking results, and acceleration functions for big data and AI.
※2 LLM (Large Language Model): A neural network with a vast number of parameters, from billions to hundreds of billions. A core technology in natural language processing and generative AI, it learns from enormous text data to enable advanced language understanding and generation.
※3 RAG (Retrieval-Augmented Generation): Technology that has a large language model reference external documents and data sources in real time when generating an answer. It helps curb hallucination (generation of false information) and improves answer accuracy.
※4 HPCG (High Performance Conjugate Gradient): A benchmark that competes on the processing speed of the conjugate gradient method actually used in industrial applications. Unlike LINPACK’s theoretical performance measurement, it is valued as a metric for evaluating more realistic computation. Fugaku has won this ranking for 10 consecutive terms (as of November 2024).
※5 CNN (Convolutional Neural Network): A type of neural network that shows high performance in fields such as image and speech recognition.
※6 GAN (Generative Adversarial Network): A mechanism that trains two networks — a generator and a discriminator — against each other, capable of generating data so refined it is indistinguishable from real data.
※7 RNN (Recurrent Neural Network): A neural network suited to handling data where “order” and “context” matter, such as time-series data and text.
※8 Google Brain: The name for the AI research project and team launched by Google. Known for training neural networks at large scale around 2012 and making a major leap in image recognition and other areas.
※9 GPU (Graphics Processing Unit): A processor originally designed to process computer graphics quickly. In recent years, because it delivers high performance in large-scale parallel computation such as deep learning, it has become indispensable hardware for supercomputers and AI research.
※10 AI for Science: An effort to apply AI not only to engineering and industry but to basic science and academic research itself, aiming for scientific discovery and the creation of new knowledge. One of the research areas R-CCS focuses on.
※11 AskDona: A generative AI chat already deployed by RIKEN R-CCS on the Fugaku support site. Using RAG technology, it answers user questions while referencing Fugaku’s manuals and technical documents.