Update:
Following my above inquiry, I ran into this post.
Based on the guidance provided, I have created a Cloudflare WAF rule that whitelists the user-agent Elfsight-AI-Chatbot-Knowledge-Crawler.
However, I still think it’s important to receive feedback from your dev team to ensure I haven’t missed any other crawlers (i.e., user-agents) that may impact the performance of the Elfsight AI Chatbot. For example, since the chatbot relies on models like gpt-5.4-mini and gpt-5.6-luna, should we also whitelist the user-agent GPTBot or any other third-party crawlers?
Finally, is this whitelisting guidance and full list of User-Agents documented anywhere in Elfsight’s Knowledge Base? Having an official Help Center article covering firewall setup and Cloudflare (i.e., CDN) compatibility for the AI Chatbot would be extremely valuable for all users.
Thank you!