We Ran AI Penetration Testing on Our Own Site. It Found a Way In

TechnologyDigital
17 Nov 2026 • 7:10 PM MYT
Gotchaa Lab
Gotchaa Lab

AI, software and tech insights for Malaysian businesses

We Ran AI Penetration Testing on Our Own Site. It Found a Way In

On 17 November 2026 we pointed an AI hacker at our own website. Shannon, an open-source AI penetration testing tool by Keygraph, ran against a copy of gotchaa-lab.com on our laptop, never the live site. Ten minutes later it had found one way in: an internal planning page that anyone with the link could read.

It's a small finding. We still missed it, and we build websites for a living.

What is AI penetration testing?

AI penetration testing uses AI agents, small programs that plan and act on their own, to attack your website the way a human tester would. They map the site, try attacks, and report the ones that worked. A penetration test, or pentest, is an agreed attempt to break in before a stranger does. (The phrase also gets used for testing chatbots, which is a different job.)

How Shannon works, in plain words

Shannon's GitHub page calls it "an AI pentester for web applications and APIs". The open-source version is free to download, but it runs on an AI model you pay for. First it reads your site's source code to spot likely weak points. Then it attacks the running site to see which are real.

Its rule is "No exploit, no report". A finding only makes the report if a working attack proved it, so you skip a scanner's long list of "possible issues".

It tests five kinds of attack: injection (sneaking commands into a form), XSS (planting a script on a page), SSRF (tricking the server into fetching something it shouldn't), broken login, and broken access rules. It has 52,000 GitHub stars as of 17 November 2026.

What it found on our site

We ran Shannon v1.9.0 on Claude Sonnet 5.5. Nine agents took turns: two to map the site, five to hunt for each attack type, one to try the access-rule attack, and one to write the report.

Shannon's run dashboard showing Completed, a duration of 10 minutes 38 seconds, and the nine agents it ranOur Shannon run on 17 November 2026, starting at 6:03 pm. Captured from our own dashboard.

The report found no login, script, injection or server-request problems. It proved one low-severity access gap: an internal planning page served to anyone, no login needed. It held no passwords, keys or customer data, and nothing could be changed through it. It was still never meant to be public, so we took it off the live site the same day.

Why so little? Our site has no logins or database, so most attacks had nothing to grab. That doesn't make our site fully safe: one 10-minute run covered five attack types, nothing more.

How this affects Malaysian businesses

The gap Shannon found needed no hacking skill: anyone with the link could read the page. Company sites collect pages like that, from old promos to forgotten test pages.

Now picture a shop or booking site with logins, orders and personal details, built by a small team or a web development partner. There, a tool like this tries to log in as someone else, read another customer's order, or slip commands into a search box. If personal data turns up, Malaysia's PDPA may also come into play.

Our take: a tool like Shannon is a cheap first pass, not a full pentest. It shows proof instead of guesses. Any team building a site with customer data should run one on a staging copy before launch.

Where AI penetration testing tools stop

AI tool like ShannonHuman penetration test
What it needsSource code and a running copyA written scope and permission
What it coversFive common attack typesWhatever is in scope, including business logic
SpeedMinutes to about 1.5 hoursSet by the scope and the tester's time
ReportProven findings onlyFindings plus a tester's sign-off

It has other limits too:

  • Only on systems you own. Shannon's README says to run it only with ownership or written permission, and "Do not run Shannon against production systems." It creates users and changes data.
  • It needs setup. Docker, the source code and a paid AI model account. That is your developer's job.
  • The AI provider can stop it. Shannon's docs note that Anthropic and OpenAI apply real-time safeguards to cyber-security work. We explain how verification for that works in our Claude Mythos access post.
  • Sites built fast with AI help need it more, for the reasons in vibe coding security risks.

When a customer or regulator wants a pentest report, you need a real one. Our penetration testing cost guide explains how to tell a real test from a scan.

What to do this week

  1. List every page on your site that is meant to be private.
  2. Open each one in a private browser window. If it loads without a login, it is public.
  3. Ask your developer when your site was last tested.

Curious how AI fits into building and checking your software? Let's chat. We'll give you an honest take, no sales pitch.

This article is for general information and does not constitute professional cybersecurity or legal advice.

References

  1. Shannon by Keygraph, GitHub README
  2. Shannon AI providers documentation
Newswav Malaysia Best News App

Newswav is an online content aggregator and obtains its content from different online sources. The content in the app do not belong to Newswav nor do they reflect the opinions of Newswav and its staff. Your use of this app indicates your understanding and acceptance of this information.

Newswav Sdn. Bhd. (201701008480 (1222645-M)) 2026 All Rights Reserved