Wijnkoperij De Gouden Ton is a Dutch wine merchant with physical stores and a Magento 2 webshop. The shop carries a large catalogue of wines and spirits for customers across the Netherlands.
Key Takeaways
| Before Botstopper | After Botstopper | |
| PHP resource overhead | Scrapers caused CPU and RAM spikes. | Down 65%, freeing CPU and RAM for real shoppers. |
| Bot traffic | Bots reached Magento directly. | Bots are blocked or challenged before they reach Magento. |
| Shopper experience | Higher TTFB and slow catalogue browsing. | Lower TTFB, faster page loads, and faster checkout. |
| Crawler control | Bots ignored the rules in robots.txt. | Unlimited allow and block rules determine which crawlers are allowed in. |
The Challenge: The Traffic Wasn’t Coming From Shoppers
De Gouden Ton’s server load started spiking, and catalogue pages slowed down. But the extra traffic wasn’t coming from people. It was coming from AI crawlers and scrapers.
These bots ignored the shop’s robots.txt file and crawled product pages, filters and layered navigation URLs over and over. Every visit triggered PHP processes and database queries, eating up server capacity that real customers needed. Genuine shoppers saw a higher TTFB and slower catalogue browsing.
The Insight: Stop Bots Before They Reach Magento
A robots.txt file only works when a bot chooses to respect it, and these bots didn’t. By the time a request reaches Magento, the server has already done the heavy lifting.
So the fix had to sit in front of Magento. It also had to be selective, letting search engines like Google through while keeping the bad bots out.
The Solution: Hypernode Botstopper
Botstopper is built into Hypernode hosting. It sits in front of NGINX and checks every incoming request before it reaches Magento.
By filtering unwanted requests before they reach Magento, Botstopper stops those bots from consuming PHP workers and database resources in the first place.
Search engines like Google and Bing are verified via reverse DNS and let through. Unknown or abusive AI bots are either blocked with a 403 or given a browser challenge. Instead of using blanket bans, Botstopper scores suspicious behaviour, so real visitors don’t get caught by mistake.
The Collaboration: Jaap Baan and Hypernode Customer Success
Jaap Baan, Webshop Manager at De Gouden Ton, reached out to Hypernode for help. Hypernode analysed the shop’s traffic patterns and worked closely with Jaap to set up Botstopper and tailor its rules to target AI scrapers.
The Results
- PHP resource overhead dropped by 65%, so CPU and RAM now go to real shoppers instead of bots.
- Page loads and checkout were back up to speed right away, with a lower TTFB for customers.
- Bot traffic is stopped before it reaches Magento’s PHP workers, protecting the shop from sudden crashes during busy times.
- Jaap’s team has full control over which crawlers get in, with no limit on how many allow and block rules they can set.
- Botstopper runs in the background, so the team can focus on the shop rather than writing custom NGINX rules to keep bots out.
Hi! My name is Dion, Account Manager at Hypernode
Want to know more about Hypernode's Managed E-commerce Hosting? Schedule your online meeting.
schedule one-on-one meeting +31 (0) 648362102
