Why I Let AI Make Suggestions, Not Decisions

As software engineers, most of us have probably experienced the pain of operational support: an endless stream of bugs, unexpected data requests, and people from other teams showing up with, “This is urgent—could you take a look first?”
In this article, I want to share an initiative we started to improve our operational workflow in the gaps between our existing development commitments.
More specifically, it is about how we introduced AI into the process without allowing it to become the final decision-maker.

Where Does the Knowledge Live?

When team resources are limited, many operational issues require people across different roles to work together. Some might say, “Software engineers do not really have fixed roles. You just handle whatever comes your way.” But as someone who transitioned into software engineering from another field, I still find myself impressed by colleagues who can handle anything thrown at them.
“Then let’s write a playbook.” That was my initial idea.

Playbook

A playbook is essentially an operational troubleshooting manual.

Based on memory, I documented the types of issues that seemed to occur frequently, along with the investigation approaches that more experienced engineers would typically take. The goal was to give others a reference when they encountered similar problems.
However, deciding what to document based purely on intuition was not exactly a scientific approach. So we brought in our new favorite colleague: AI.

We collected a set of historical issues and asked AI to identify the most common problem patterns. We then wrote documentation for those high-frequency cases.
But what about operational issues that were not covered by the playbook?
For those, we still had to rely on memory and experience.

“I feel like we’ve seen something similar before.” With only a vague recollection, we would search through team discussion records using keywords, hoping to find a previous issue with a similar cause and reuse its solution. This approach worked reasonably well for long-tenured team members, but for new hires, almost every issue was new. The knowledge had not disappeared; it was simply locked inside individual people’s memories.
We also discovered that these “high-frequency issues” accounted for only around 10–20% of all issues. The product evolved too quickly, and documentation became outdated just as fast. Writing more playbooks was not going to be the most effective solution.

A Playbook Alone Is Not Enough

More importantly, when the real problem is too many issues and too few people, the answer should not be to turn every team member into an engineer who can resolve issues at superhuman speed. The better solution is to reduce the number of issues at the source.
That may sound idealistic—almost like a utopia for software teams—but we decided to try anyway.

Postmortem Notes

We decided that every resolved issue should leave behind a structured postmortem record. It would include the cause of the issue, a high-level summary of the investigation, and areas that could be improved afterward.
The problem was that some issue discussions contained more than a hundred messages. Reading the full context and extracting the important details took a significant amount of time, so once again, we called on our AI colleague.

Fully Automated vs. Human-in-the-Loop—and the Cost of Human Review

Our original goal was to have AI analyze a large batch of issues at once, generate postmortem notes and follow-up improvement areas, and write everything back into our existing internal record system.

However, because this information would later be used to determine improvement priorities, we were not comfortable allowing AI to make all of those decisions on its own. Incorrect information can be more dangerous than having no information at all. We therefore decided that AI would first present its results for review, and only write them into the internal system after a human had confirmed them.

Once we started running the workflow in practice, we found that reviewing both the generated postmortem notes and the proposed improvement areas was not as lightweight as expected. For short and simple issues, verification took very little time. But for complex issues, we still had to read through more than a hundred messages before we could determine whether the AI-generated summary was accurate.

In other words, we ended up spending much of the time that AI was supposed to save.

We therefore needed a systematic way to measure the reliability of AI-generated output.

When Information Is Missing, Say So—Preventing Fabrication Matters More Than Making the Model Smarter

At first, we simply asked AI to read the discussion history and analyze each issue. When we tested the workflow on a small number of issues, there were no obvious problems, and most of the output seemed reasonable.
However, when we increased the number of issues processed in a single batch, hallucinations began to appear.

For example, a discussion might only say that a follow-up improvement should be made later, but the postmortem note would claim that someone had already completed the fix. In other cases, the model would confidently say something like, “Because this feature is implemented in a particular way, it can be improved using this approach,” even though the model had never been given the relevant source code.

When working with AI, you have to state the rule explicitly: say what you know, and admit what you do not know.
We updated the prompt to require the model to clearly state when there was not enough information to complete the analysis. Otherwise, fabricated details would be written into the postmortem notes and become noise rather than useful knowledge.

Six Fixed Improvement Categories Instead of Open-Ended Classification

For follow-up improvement areas, we deliberately avoided letting AI generate categories freely.

Anyone who has worked with AI has probably encountered the same issue: an LLM does not retain memory across independent sessions. If we asked it to define categories on its own, one batch might produce one taxonomy while the next batch produced another. Even when the underlying concepts were the same, small wording differences would make aggregation and analysis difficult after only a few dozen records.

After discussing the problem with AI, we settled on six fixed improvement categories and wrote an initial definition for each one. The model could then classify issues according to those definitions.

The “Add to Playbook” category also gave the original playbook a new role. Instead of deciding what to document based on intuition, we could now use postmortem classification results to identify suitable candidates. Only recurring problems with clear and repeatable procedures would be added.

The playbook still existed, but it was no longer the primary strategy. It became a downstream output of the postmortem workflow. This prevented it from becoming an oversized collection of unrelated information that was difficult to search.

Meanwhile, one-off issues that were not suitable for an SOP remained available as structured, searchable postmortem notes instead of living only in the minds of senior employees.

This was the prompt we used at the time:

You are a professional technical issue analyst. Based on the following team discussion history, analyze the issue and provide:

1. Postmortem Note: In 1–3 sentences of Traditional Chinese, explain:

   • What the problem was
   • The root cause
   • How it was resolved, if it has been resolved

   If there is insufficient information, explicitly state: “Insufficient information to complete the analysis.”

2. Follow-up Improvement Areas: Select all applicable categories from the fixed options below. Multiple selections are allowed, and the result may also be an empty list.

   • Add to Playbook: Repetitive manual procedures with a clear process that are suitable for an SOP, or complex investigation scenarios that would benefit from an engineering troubleshooting guide.
   • Product and Technical Improvements: Problems with opportunities for systematic or structural improvement.
   • Add Logging: Insufficient log data makes troubleshooting difficult or reduces system transparency.
   • Monitoring: Proactive alerts are needed to prevent failures from occurring silently, such as interrupted data synchronization.
   • External Data Specifications: The source data format, synchronization specification, or data quality does not match expectations.
   • User and Operational Experience: Problems caused by workflow design, unclear interface guidance, or gaps in user understanding.

Reliability Comes from Constraints—but Constraints Have a Cost

Using the approach described above, we analyzed nearly 200 historical issues.

Based on the resulting postmortem notes and improvement categories, we then asked AI to use the available technical context to propose an initial set of ten follow-up improvement initiatives. We planned to gradually add them to our development roadmap.

However, some of those ten proposals were still reported as inconsistent with the current state of the system. We are therefore continuing to refine the workflow.

For an internal tool like this, reliability does not come primarily from having the most capable model. It comes from three constraints:

• Humans retain control over what gets written into the system.
• The model must classify results within a closed and predefined category space.
• The model must acknowledge uncertainty when the available information is insufficient.

Even so, these constraints are not free.

Human review may provide a relatively stable safeguard, but its throughput does not scale, and different reviewers may reach different conclusions. A more sustainable path would be to gradually move from “a human reviews every output” toward “the system measures output quality systematically.”

The next article will discuss how we are building an evaluation framework for this internal tool. That framework is still under development, though, so perhaps I will see you again in a few months.