GFLOPS Co., Ltd. (“GFLOPS”) and JSOL Corporation (“JSOL”) have been running a joint validation to improve the accuracy and standardization of system risk assessment work, using “AskDona®,” GFLOPS’s enterprise generative AI platform built on high-precision RAG (Retrieval-Augmented Generation). As partners, GFLOPS and JSOL now plan to fully launch AskDona “Batch Assessment” — a feature powered by an AI assessment agent that streamlines assessment work such as system risk assessment.
Background
System risk assessment is the practice of investigating the risks latent in an information system and evaluating their impact. Financial institutions in particular must identify and evaluate the risks in their own systems based on the FSA’s “Guidelines for Cybersecurity in the Financial Sector.”
As a member of the NTT DATA and SMBC groups, JSOL conducts a system risk assessment of the 400 items defined by the SMBC group once a year for the system services it plans and sells and for its internal business systems, and implements risk measures based on the results. This placed a heavy burden on staff, requiring roughly 4,300 hours (about 570 person-days) per year.
Specific challenges
- Interpretation and evaluation of assessment items varied from person to person
- Reading and evaluating 400 assessment items and criteria, searching for and excerpting relevant descriptions, and attaching supporting evidence involved an enormous amount of verification work
About the validation
In this validation, GFLOPS’s RAG was applied as a means of retrieving the right information from the multiple internal documents related to JSOL’s internal business systems. For the assessment items JSOL defines in advance, an AI agent investigates appropriate evidence from AskDona’s RAG database and performs the evaluation. By having AI handle the first-pass evaluation of system risk assessment work that had long depended on manual effort, the aim was to make the work more efficient.
AskDona, GFLOPS’s RAG-based generative AI platform, applies Agentic RAG technology that generates highly comprehensive answers. Because it delivers high answer accuracy without modifying the internal data registered in the RAG database and requires no RAG tuning, the RAG provided by AskDona (“AskDona RAG”) was used as the core technology for this validation.
The validation was conducted in two main phases — a “technical validation phase” and a “business application phase” — from April to October 2025.
In the technical validation phase, design and operational documents for more than 100 target systems were registered in the AskDona RAG database, and target/out-of-scope conditions were set for roughly 400 assessment items according to each system’s characteristics. The accuracy of the AI agent’s evidence-based evaluations was then verified.
In the business application phase, AskDona “Batch Assessment” — powered by GFLOPS’s AI agent and refined through the technical validation — was applied to JSOL’s actual workflow in line with this year’s system risk assessment. This confirmed how much JSOL employees felt their work had become more efficient, its contribution to process improvement, and the effect on workload and time reduction.
Results
In the technical validation phase, the AI agent’s accuracy was verified against the results of JSOL’s vulnerability assessment from the previous year. Across the four-way classification of “addressed,” “not addressed,” “out of scope,” and “cannot be determined,” accuracy of over 90% was confirmed. For items lacking the information needed for evaluation, the system does not infer a result; instead it returns “cannot be evaluated” and presents the information that is missing.
In the business application phase, having AskDona “Batch Assessment” perform the first-pass evaluation — interpreting assessment items, evaluating them, and recording the results — reduced effort by an average of 45% per system and by roughly 2,000 hours (about 260 person-days) per year across JSOL. Time spent on system risk assessment, which had been roughly 4,300 hours (about 570 person-days) per year, was confirmed to fall to roughly 2,300 hours (about 300 person-days).
Introducing AskDona “Batch Assessment”
Batch Assessment is a feature in which, for a large number of predefined assessment items, an AI assessment agent automatically makes judgments based on the documents and regulations referenced by AskDona RAG.※ Based on evaluation criteria set by an administrator, it evaluates classifications such as “addressed,” “not addressed,” and “out of scope” in bulk, and presents the evidence and referenced passages behind each judgment. When information is insufficient, it does not infer but marks the item “cannot be determined” and states what is missing. In assessment work that tended to depend on individual experience and interpretation, it supports the unification of judgment criteria and clarity of the evaluation process.
Assessment work proceeds through the following steps:
- Select the systems to be assessed
- Tailor the inspection items
- Interpret the inspection items for the target system
- Collect supporting evidence
- Record and evaluate inspection results
- Formulate an action plan
By changing the evaluation criteria and the knowledge referenced, it can be flexibly applied to a wide range of assessment work such as IT control evaluations and compliance checks. The evaluation mechanism can be reused to match the rules of each task, contributing to efficiency across many domains.
AskDona “Batch Assessment” from GFLOPS will be fully launched toward next fiscal year, with JSOL as a joint sales partner.
※AskDona RAG handles source retrieval and evidence presentation, and the AI assessment agent makes judgments based on that information.
About GFLOPS Co., Ltd.
GFLOPS provides enterprise RAG solutions under the philosophy of “let AI handle the work AI can do, so people can focus on the work only people can make valuable.” Aiming to build a secure foundation that can also handle highly confidential documents, it focuses on AI model development and its own RAG architecture.
- Company: GFLOPS Co., Ltd.
- Representatives: Maria Morimoto (CEO); Ryosuke Suzuki (Co-Representative)
- Head office: Shibuya-ku, Tokyo
- Business: Development and provision of AI services leveraging large language models (LLMs) and generative AI technology
- Website: https://gflops-ai.com/
About JSOL Corporation
JSOL Corporation is a system consulting and solution integrator funded by NTT DATA Corporation and The Japan Research Institute. Since its founding in 2006, it has drawn on deep business know-how and system development capabilities cultivated in manufacturing, distribution and services, finance, and the public sector, along with advanced analysis technology in engineering science, to contribute to raising customer value.
- President & CEO: Takeshi Nagai
- Head office: Kudan Kaikan Terrace, 1-6-5 Kudanminami, Chiyoda-ku, Tokyo
- Web: https://www.jsol.co.jp/