We code with AI every day. Two people would never have built our own invoicing and operations system InvoAIM any other way — and there’s no point pretending it can be done without it. Which is exactly why we know where it goes wrong. And it doesn’t go wrong where people expect.
What AI actually gets wrong in code
People expect AI to produce typos and nonsense. It doesn’t. The code it writes compiles, the tests pass, and the demo works. The mistakes it makes are of a different kind:
- Missing permission checks. A function returns an order by ID. Nobody wrote down that it should be the logged-in user’s order — and AI won’t add that on its own.
- Exposed interfaces. An admin endpoint was created during debugging and stayed reachable without authentication.
- Keys in the repository. An API token got hardcoded into a file while testing. It was later moved to an environment variable, but it stayed in the commit history.
- Outdated dependencies. The model suggests a library it knows from its training data — that is, one that was popular two years ago.
None of these show up when you skim the code and think „looks fine". You have to go looking for them deliberately.
Numbers, not impressions
This isn’t a hunch that „AI code is worse". Measurements from 2025 say this:
- In 45% of tasks, AI-generated code contained a security vulnerability. Over 100 models were tested on 80 tasks — Veracode.
- Repositories written with AI assistance show up to ten times more security findings than the rest — Apiiro.
The other half of the equation is volume. Google and Microsoft both state publicly that AI now generates roughly a quarter to a third of new code, and at some younger companies it’s the majority. Do the multiplication.
What’s telling is that the error rate per task hasn’t improved much over earlier models. Models can do more and write more readable code — but they add security checks only when asked. Better capabilities won’t solve this problem on their own.
Why tests don’t catch it
Tests verify that an application does what it’s supposed to. Security is the opposite — what happens when someone uses it in a way you never intended:
| What we test | What we don’t test |
|---|---|
| a user views their own order | a user views someone else’s order |
| a logged-in admin opens the admin panel | the admin panel is opened by someone not logged in |
| an image gets uploaded | a script renamed to look like an image gets uploaded |
| the form saves a name | the form saves a chunk of an SQL query |
Green tests and a wide-open admin panel coexist without the slightest difficulty. Neither tells you anything about the other.
From 11 September 2026 this changes on paper too
Until now, having code reviewed was voluntary diligence. The Cyber Resilience Act turns it into an obligation with a deadline.
EU Regulation 2024/2847 applies to products with digital elements placed on the EU market. What matters is whether you supply software onward, not how big you are:
- 11 Sep 2026 — reporting obligations begin. An actively exploited vulnerability or a severe incident must be reported: an early warning within 24 hours, a full notification within 72 hours, a final report within a month.
- 11 Dec 2027 — the rest applies: security across the whole lifecycle, technical documentation, conformity assessment and CE marking.
For a vulnerability to be reportable within 24 hours, someone has to be able to find it and describe it. That’s not something you sort out at the moment it happens.
On top of that comes NIS2, implemented in the Czech Republic as Act No. 264/2025 Sb. (Czech collection of laws), which affects roughly six thousand companies. It puts pressure on the supply chain — if you supply software to a regulated company, the scrutiny comes from them, whether the regulation applies to you directly or not.
What to do about it in practice
If you don’t want to buy anything, you can handle this yourself, and it has the best benefit-to-time ratio:
- Go through the commit history for keys and passwords. Not just the current state — the history. A token you find has to be revoked, not deleted.
- List all your endpoints and answer this for each one: who can reach it without logging in?
- For every read of data from the database, verify that it filters by the logged-in user, not just by an ID from the request.
- Update your dependencies and go through known vulnerabilities.
- Turn off debug mode and error messages that print the application’s internals.
Once you’re done, the other half remains: documenting it. A mental note is no evidence for a client, nor for documentation under the CRA.
When to call someone from outside
Your own code is hard to read — you know how it was meant to work, so you read what you intended rather than what’s actually there. An outside view makes sense mainly when the application handles personal data or payments, when you’re handing it over to a client, or when someone asks you to prove it has been checked.
We offer a pre-launch application review: automated scanning, manual verification of findings, and a written report with a verdict. We process code locally in the Czech Republic, not in a third-party cloud — with source code that tends to be a requirement, not a whim.
Frequently asked questions
Does this mean you shouldn’t code with AI?
No. We use it daily ourselves, and we couldn’t have built our own system at this pace without it. The problem isn’t the tool — it’s the ratio between how much code gets written and how much of it anyone reads. Typing speed used to hold that ratio in check. Today nothing does, so it has to be restored deliberately.
Won’t the tests catch it?
Usually not. Tests verify that an application does what it’s supposed to when a well-behaved user uses it. Security flaws are about what happens when someone uses it differently: calls someone else’s endpoint, sends a different ID, tries to upload a file of another type. Green tests and a wide-open admin panel coexist perfectly happily.
Does the Cyber Resilience Act apply to small companies too?
What matters is whether you place a product with software on the EU market, not how big you are. Reporting obligations for actively exploited vulnerabilities and severe incidents apply from 11 September 2026, the rest from 11 December 2027. The deadlines are 24 hours for the early warning and 72 hours for the full notification.