Requirement Definition, Authority, and Approval Design When Major TMT Companies Introduce Robo Claw
Once the target tasks are determined, concretize the As-Is/To-Be, inputs and outputs, reference data, system connections, access rights to the production environment, Secret management policy, approvals, responsibility boundaries, logs, and KPIs.
The purpose of the Refine STEP is to document "who will automate which parts, to what extent, and under which approvals" for the target tasks. In particular, make the production access rights, Secret management policy, and approval conditions for Pull Requests or external transmissions clear at this stage. Proceeding to the pilot with ambiguity here will cause redesigns in later stages.
Who This Is For
Who This Is For
This is targeted at personnel from the information systems department, security department, development department managers, and internal audit department who have determined the target operations in Discover STEP.
What You'll Decide
What You'll Decide in This Step
In Refine STEP, the scope of the target operations, data handled, connected systems, access rights to the production environment, Secret management policy, approval conditions, responsibility boundaries, and KPIs are documented and translated into requirements that can be verified in Build & Validate.
Industry Challenges
Issues specific to major TMT companies
The concept of permissions differs for each product.
Existing permission operations vary by product and team, making it difficult to design unified permissions.
Rules for Secret management are not established.
In many cases, rules on who manages API keys and tokens and to what extent are not clearly documented.
PR review approval criteria are dependent on the individual.
Whether a review comment can be adopted and merged depends on the personal experience of the reviewer.
Responsibility boundaries tend to be unclear.
The allocation of responsibility between AI agents, systems, and humans when failures or errors occur is not organized.
Method
Implementation Steps
1. Organize As-Is/To-Be
We organize by comparing the current business flow with the business flow aimed for after the introduction of Robo Claw.
2. Define the target scope and inputs/outputs.
Clearly define the scope of the target tasks and specify the input data and outputs (draft review comments, draft responses, updates, etc.).
3. Design Agents, Skills, and Tools
Design the roles of Agents required for the target tasks, the Skills for business procedures, and the scope of Tools that perform external operations.
4. Define the Tool Policy
Define as Policies, per repository/environment, the operations that Tools are allowed or denied to execute (Allow/Deny).
5. Design access permissions for the production environment
Design who can use or modify Agents to what extent in development, testing, and production environments.
6. Define secret management policies
Define policies for issuing, storing, and rotating API keys, tokens, etc.
7. Define approvals and escalations
Define approvers for Pull Request merges, approvers for external transmissions, and escalation destinations in case of exceptions.
8. Define responsibility boundaries, logs, and KPIs
Define the responsibility boundaries between AI agents, systems, and humans, the scope of operation logs to be preserved, and KPI for effect measurement.
Data & Systems
Data and Systems Used
Human-in-the-loop
Where Human Approval Is Required
- Clearly specify Pull Request merge approvers by position or team
- Clearly define the final approver for external communications with customers by position.
- Obtain approval from the Information Systems Department for the issuance and rotation of secrets.
- Obtain approval from the Information Systems Department for granting permissions across product teams.
Measurement
KPI
In the Refine STEP, define candidate KPIs to be verified in Build & Validate
Target level for time spent creating review comments.
Target time for drafting review comments for Pull Requests.
Lead time until approval
Time required from draft creation to human approval
Number of detected permission deviations
Number of detected operations exceeding the defined permission scope
Checklist
Requirements definition checklist
- The scope of the target work (included and excluded tasks) is documented.
- Input data and outputs (review comments, draft responses, updates) are defined.
- Systems to be connected and whether they can be connected are confirmed by the Information Systems Department.
- Access permissions for development, verification, and production environments are defined.
- Policies for issuing, storing, and rotating secrets are defined.
- Approval conditions and approvers for Pull Request merges and external communications are defined.
- Escalation destinations and flow in case of exceptions are defined.
- Responsibilities between AI agents, systems, and humans are documented.
- The scope and retention period of operation logs are defined.
- KPIs used for effect measurement are defined
Pitfalls
Common Pitfalls
Postpone authority design.
If you try to adjust permissions after running first, significant rework occurs when deploying to multiple products.
Proceed with the approver unclear
Deciding approvers by individual name rather than position will cause operations to stop during transfers or resignations.
Decide KPIs based on effectiveness.
If you set optimistic numbers as KPIs, evaluations in Build & Validate will become superficial.
FAQ
Frequently Asked Questions
Which department should lead the requirement definition?
It is recommended that the field managers of the target business (development, QA, SRE, etc.) and the Information Systems department jointly lead, and that the Security and Legal departments also be involved if external transmission is involved.
How detailed should authority design be?
At a minimum, clarify the access scope for development, testing, and production environments, as well as the approvers for Pull Request merges and external transmissions. The level of detail should be adjusted according to the risk of the target business.
What happens if too much time is spent on Refine?
Trying to perfect the requirements will delay the start. It is recommended to first solidify requirements that can be verified at a pilot scale and proceed on the premise of reviewing them when expanding to production.
Shall we organize the requirements definition and access permissions together?
The scope of the target business, permissions, approval flows, and responsibility boundaries can be concretized through consultations in the official LP.