# Issue with web pages training

**URL:** <https://community.elfsight.com/t/issue-with-web-pages-training/173461>\
**Category:** General Questions\
**Tags:** ai-chatbot\
**Created:** [September 4, 2026, 2:29pm UTC](https://community.elfsight.com/t/issue-with-web-pages-training/173461 "2026-09-04T14:29:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Markos](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@Markos](https://community.elfsight.com/u/Markos)\
**Post date:** [September 4, 2026, 2:29pm UTC](https://community.elfsight.com/t/issue-with-web-pages-training/173461/1 "2026-09-04T14:29:44Z")

</div>

**Hello,**

**I think I am facing the exact same issue. Could you please take a look at my widget as well?**

**Details:**

- **Widget ID:** `662cf189-0c9c-4800-a48e-d3f3d494d5b6`

- **Domain:** `apogee.gr`

- **Affected URLs:**

- **Firewall / CDN status:** We have verified in Cloudflare logs that requests from the `Elfsight-AI-Chatbot-Knowledge-Crawler` User-Agent are passing through with a `200 OK` / `Skip` status, so Cloudflare is not blocking the crawler.

**Could you check the backend logs for my widget to see why the scraping fails with the red alert?**

**Thanks!**

 ![image](https://us1.discourse-cdn.com/flex020/uploads/elfsight/original/3X/3/f/3f4f38f9c9db68e513d34842251f9251d5753261.png)

---

<div class="post-metadata">

**Author:** ![Max](https://sea2.discourse-cdn.com/flex020/user_avatar/community.elfsight.com/max/32/710_2.png) [@Max](https://community.elfsight.com/u/Max)\
**Post date:** [September 4, 2026, 3:30pm UTC](https://community.elfsight.com/t/issue-with-web-pages-training/173461/3 "2026-09-04T15:30:59Z")

</div>

Hi there, @Markos 👋

I am so sorry about this issue!

Please let me check it with the dev team. I’ll report back once I have their response 🙂

---

<div class="post-metadata">

**Author:** ![Max](https://sea2.discourse-cdn.com/flex020/user_avatar/community.elfsight.com/max/32/710_2.png) [@Max](https://community.elfsight.com/u/Max)\
**Post date:** [September 7, 2026, 10:45am UTC](https://community.elfsight.com/t/issue-with-web-pages-training/173461/4 "2026-09-07T10:45:44Z")

</div>

Hi @Markos 👋

Thank you for waiting!

Our devs looked into the issue, and it appears that your website’s bot protection (most likely Cloudflare or another WAF) is occasionally blocking requests from our crawler and returning a `403 Forbidden` error.

To allow the AI Chatbot to access and process the page, please add an exception/allow rule in your Cloudflare or WAF settings for our crawler’s User-Agent:

`Elfsight-AI-Chatbot-Knowledge-Crawler/1.0 (+https://elfsight.com)`

Full User-Agent string:

`Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 Elfsight-AI-Chatbot-Knowledge-Crawler/1.0 (+https://elfsight.com)`

Once the crawler is allowed, the link should be added and processed correctly. Check it out and let me know if it helped 🙂