AI tools in software development range from simple API calls to fully autonomous agent systems. Understanding the categories, their risk profiles, and governance requirements is essential for any organisation adopting AI-assisted development.
AI Technology Categories
LLM APIs
Large Language Model API providers (OpenAI, Anthropic, Google AI, Cohere, Mistral).
- Risk Level: Medium
- Governance: May process sensitive data via API calls. Requires data classification review.
AI Coding Assistants
AI-powered code generation and assistance tools (GitHub Copilot, Cursor, Codeium, Tabnine, Amazon CodeWhisperer).
- Risk Level: Medium
- Governance: May expose code context to external services. Requires security review.
AI Frameworks
AI/ML application frameworks and orchestration (LangChain, LlamaIndex, Haystack).
- Risk Level: Low
- Governance: Standard dependency management applies.
Vector Databases
Vector storage for embeddings and RAG applications (Pinecone, Weaviate, Qdrant, ChromaDB, pgvector).
- Risk Level: Low
- Governance: Document embeddings may contain sensitive content.
AI Agents
Autonomous AI agent frameworks with tool use capabilities (LangChain Agents, CrewAI, AutoGPT, Claude Agent SDK).
- Risk Level: High
- Governance: Agents can execute code, access files, and make network requests autonomously. Requires strict resource limits.
Agent Orchestration
Multi-agent coordination and workflow systems (LangGraph, CrewAI Crews, AutoGen GroupChat).
- Risk Level: High
- Governance: Multi-agent systems multiply attack surfaces. Require strict resource limits and audit logging.
Usage Patterns and Maturity
| Pattern | Maturity | Key Indicators |
|---|---|---|
| Simple API Call | Experimental | Single API call, basic error handling |
| Tool/Function Calling | Emerging | tools= parameter, function definitions |
| Basic RAG | Emerging | Vector store queries, context assembly |
| Advanced RAG | Standardised | Multiple retrievers, reranking |
| Single Agent | Standardised | Agent executor, tool chain |
| Multi-Agent System | Optimised | Agent orchestration, role specialisation |
| Autonomous System | Strategic | Continuous execution, goal decomposition |
Maturity Levels
| Level | Name | Description |
|---|---|---|
| 1 | Experimental | Individual developers testing AI APIs. Hardcoded keys, single-file usage. |
| 2 | Emerging | Team-level AI adoption. Environment variables, basic retry logic. |
| 3 | Standardised | Organisation-wide AI standards. Shared libraries, proxy/gateway usage. |
| 4 | Optimised | AI Centre of Excellence. Custom abstractions, caching layers, usage analytics. |
| 5 | Strategic | AI-native architecture. Multi-model routing, agentic workflows, RAG infrastructure. |
Governance Framework
Categories Requiring Approval
- LLM APIs (
ai-ml/apis) - AI Agents (
ai-ml/agents) - Agent Orchestration (
ai-ml/patterns/orchestration)
Categories Requiring Security Review
- LLM APIs — data classification and API key management
- AI Agents — autonomous execution capabilities
- Tool/Function Calling — input validation requirements
- AI Coding Assistants — code context exposure
Data Classification
| Category | Risk |
|---|---|
| LLM APIs | May process sensitive data via API calls |
| AI Agents | Autonomous execution with external access |
| RAG | Document embeddings may contain sensitive content |
| Coding Assistants | May expose proprietary code to external services |
Risk Mitigation
For AI Agents
- Implement least-privilege tool access — Only grant the permissions each agent needs
- Add rate limits and circuit breakers — Prevent runaway execution
- Log all agent actions for audit — Every tool call, every decision, every output
- Implement kill switches — Ability to terminate autonomous agents immediately
For Tool/Function Calling
- Validate all tool inputs — Never trust LLM-generated arguments without validation
- Avoid eval/exec with tool arguments — Treat tool inputs as untrusted user input
- Implement rate limiting — Prevent excessive tool calls
For Agent Orchestration
- Set iteration and time limits — Prevent infinite loops
- Authenticate agent-to-agent communication — Don't assume internal messages are trustworthy
- Monitor resource consumption — CPU, memory, network, and API call budgets
Tracking AI Tool Adoption
Most organisations have no visibility into which AI tools developers are using. Build inspection and desktop monitoring can identify:
- Which AI coding assistants are in use (and which are unsanctioned)
- How much code is AI-generated vs human-written
- Which teams are early adopters and which need support
- Whether AI-generated code follows the same review standards as human code
This visibility is the prerequisite for effective governance. You cannot govern what you cannot see.