BUILD / AI & Software
So you’re the AI person at work now.
A quick guide of what you should know before connecting company data, systems, or customers to your first AI automation.
In this new world of AI, it's important to understand the risks and how to mitigate them.
I’ve spent my career wishing more people outside engineering would give tech a try. Or at least be interested enough or know enough for me to complain about a hectic day in the life of me a software engineer, where i didnt feel like i needed to get a white board out just to explain enough of the basics so the alert that went off that ruined my day could be understood. understand the systems around them and feel confident enough to build something useful instead of waiting six months for it to reach a roadmap.
So if you have become the person who knows how to make ChatGPT, Claude, Copilot, an automation platform, or an AI agent do useful work, this guide is for you.
You do not need to become a software engineer before you are allowed to build. You do need to understand where the security boundaries are, because powerful systems can now be connected remarkably quickly. AI has made building easier. It has not made security optional.
01 / The adoption gap
The problem isn’t that people are using AI.
The problem is that organisations are often adopting it faster than they can govern it. IBM’s 2025 Cost of a Data Breach research found a large gap between AI use and the policies and access controls around it.
Shadow-AI incidents involved personally identifiable information in 65% of cases and intellectual property in 40%.Read IBM’s 2025 findings
That equation can happen anywhere. A developer can commit a secret. A security team can misconfigure identity. A marketer can paste a confidential document into a consumer account. A consultant can connect a third-party workflow to production. Everyone gets a turn at making the mistake.
02 / The data boundary
Can AI leak company data?
Yes—if company information is pasted, uploaded, retrieved, logged, or exposed through an integration.The risk depends on the tool’s data terms, retention, access controls, training policy, connected systems, and the data you provide. Treat every AI workflow as a data flow, not as an isolated chat window.
What data should you never put into an AI tool?
File format is not a security classification. “It’s just a PDF” and “it’s only an internal spreadsheet” tell you nothing about the consequences of exposure.
| Category | Examples | What to do instead |
|---|---|---|
| Customer data | Names, contact details, support histories | Use approved, access-controlled systems and redact test data. |
| Regulated data | Financial, health, payroll, identity information | Require privacy, legal, and security review before processing. |
| Company knowledge | Contracts, strategy, internal documents, source code | Use the approved enterprise environment and the smallest context. |
| Credentials | Passwords, API keys, cookies, certificates | Keep secrets outside prompts and manage them in a secrets system. |
Employees reportedly entered confidential source code and internal information into ChatGPT while using it for work. Samsung temporarily restricted generative-AI use while it developed additional controls. This was not an exotic model attack. Information was simply placed into a tool without the boundary being understood.
Read the reported Samsung responseA useful rule: if you would not email the material to a random external company, do not automatically paste it into an AI tool. Find the approved environment and understand its controls first.
03 / Access
A chatbot answers. An agent acts.
There is a significant difference between “summarise these documents” and “read our documents, check the CRM, update customer records, send emails, and issue refunds.” The second is not simply an assistant. It is a system with permissions.
- Read
See selected information.
- Draft
Prepare text or proposed changes.
- Send
Communicate or trigger another system.
- Change
Delete, refund, purchase, approve, or modify records.
What can it read?
Documents, customer records, messages, source code, logs, or financial data?
What can it write?
Drafts, CRM fields, databases, files, tickets, or account settings?
What can it cause?
Emails, refunds, purchases, deletions, approvals, or production changes?
Give it the smallest possible permissions.
Read-only access beats admin access. Drafting beats automatic sending. One field beats the entire database. If the agent does not need payroll, do not give it payroll.
05 / Vendors & validation
Treat your AI vendor like a vendor.
An AI provider, automation platform, vector database, CRM, browser extension, MCP server, or third-party API creates a data flow. You do not need to understand every line of its infrastructure, but you do need to know where company information goes.
Every arrow needs an owner, a reason, and a way to revoke access.
- What information are we sending?
- Where is it processed and who can access it?
- Is it retained or used for model training?
- Can we delete it and verify deletion?
- What permissions does the integration receive?
- What happens when we disconnect it?
Do not assume the AI is telling the truth
AI can produce convincing nonsense. When its output becomes another system’s input, one mistake can turn into a small automated misinformation factory.
A good workflow does not ask only, “Did the AI return an answer?” It asks, “Is this answer valid enough to safely trigger the next action?”
06 / The fundamentals
Protect the boring stuff.
AI security conversations become futuristic very quickly. Some of the most serious incidents still begin with very ordinary failures.
Researchers found weak default credentials and an API access- control flaw in the AI-powered hiring platform used by many McDonald’s franchisees. The flaws could have exposed records associated with tens of millions of applicants. The model did not need to “go rogue”; ordinary authentication and authorisation failures created the risk.
Read the McHire investigationGoogle Cloud’s Mandiant report makes a similar point: many of the most pressing AI security problems are foundational gaps in governance, visibility, testing, and IT hygiene around the AI implementation—not a novel model exploit.
And yes, developers mess this up too. Experienced engineers have hopefully learned to ask about trust boundaries, credential ownership, blast radius, revocation, auditability, malicious input, failure, and recovery. Those are systems questions. They are not exclusive to programmers.
07 / Five minutes
The five-minute AI risk check
Before launching an AI automation at work, write down five answers. If you cannot answer them, you have found the part that needs engineering, privacy, legal, or security review.
Map data, access, actions, failure, and recovery before an AI workflow touches company systems.
- 01Data
What information does it see?
- 02Access
Which systems can it reach?
- 03Actions
What can it change, send, delete, or purchase?
- 04Failure
What happens when it gets something wrong?
- 05Recovery
Can we undo what it did?
Unknown is not a “yes.” It is a review step.
08 / A safe start
How do I safely introduce AI automation at work?
Start with one useful, low-consequence workflow.Use approved tools, minimise the data and permissions, require human review, validate outputs, log actions, and prove that you can revoke and recover. Expand only after observing the real workflow—not the optimistic demo.
- 01
Name one accountable owner for the workflow.
- 02
Classify the data before connecting it.
- 03
Use an approved provider and organisation-owned account.
- 04
Give every integration the least possible permission.
- 05
Keep high-impact actions behind human approval.
- 06
Validate AI output before another system consumes it.
- 07
Log inputs, decisions, actions, and approvals appropriately.
- 08
Test failure, revocation, and recovery before launch.
- 09
Start with a small cohort and review what actually happens.
The goal is not to stop people building
Blanket bans can push useful experimentation deeper into shadow AI and further from the organisation’s security controls. A healthier model gives people an approved path, clear policy, training, and enough freedom to solve the business problems they understand best.
Google Cloud’s guidance recommends visibility and enterprise- ready controls rather than pretending demand will disappear. Its current Mandiant research also describes governance as a way to help teams innovate with more confidence, not merely a brake.
What is the smallest useful thing I can automate safely?
Build that. Watch it. Test it. Give it less access than you think it needs. Keep a human in the loop while you are learning. Then expand.
AI is lowering the barrier to building software. I think that is genuinely fantastic. Let’s lower the barrier to building useful things without lowering the barrier to accidentally giving a chatbot the keys to the company.
Further reading & primary guidance