We’d like to provide a website URL, and the assistant will scrape the content to create knowledge base.

I urgently need the same functionality. Either the bot reads the data directly from the website and all its subpages, or it searches a created XML file with the URLs for its results.

Hey there and welcome to the Community :waving_hand: Many thanks for pointing us in this direction! We see the interest from the Community in this feature and understand its importance for the users. Hopefully the devs will be to consider this request in the future. We’ll keep you posted here in c…

You need a sitemap that lists the links to all the pages you want to include for the bot. Browse to YOURSITE.COM/SITEMAP to see if it already exists. If not, your web hosting company support should assist you in finding or creating one. I’ve used Visual Studio Code with Git Hup Co-pilot to write py…

I tried the AI chatbot and thought this is already a base feature. Was surprised it is not! If it is there I go for it.

Greetings, @Tom_Fuchs and welcome to the Community :waving_hand: Many thanks for the feedback! I completely understand your point. Let’s hope the devs will release it as soon as possible and we’ll keep you posted here on any progress :slightly_smiling_face:

Attached is a python file I developed with GITHUB AI to scrape a website sitemap (best use: website content pages sitemap). Load the file into Visual Code or another python editor. Running the code will prompt you for the website URL address and generate both a JSON file and a text file of the sitem…

Hola Salvatore, no veo el archivo adjunto, me lo puedes enviar, es una pasada lo que has creado y tengo ganas de probarlo Mucha gracias El El mié, 11 jun 2025 a las 21:48, Salvatore Salvia via Elfsight Community < notifications@elfsight.discoursemail.com > escribió:

Robofi.ai looks interesting. But my next step is to build my own chat bot and own it. Sal “Tory” Salvia [image] Home Explore our collection of sailing documentaries and how-to sail and maintain your sailboat videos. Sailflix.com 301-641-3589 (m) s/v Sparkle…

Max, We previously tested a competing AI chatbot that used both website crawl data and uploaded training files. Unfortunately, we found it impossible to train that system to give valid responses. The competitor’s AI consistently confused our products and mishandled overlapping information between …

URL-based data scraping to create a knowledge base

Wishlist AI Chatbot

Topic		Replies	Views
AI Chatbot: Train your assistant using webpage URLs 🚀 New Feature ai-chatbot	5	583	September 24, 2025
Automatically crawl all pages within a website to a knowledge base AI Chatbot Gathering-Feedback	0	48	September 24, 2025
Training AI Chat Bot with sitemap AI Chatbot Released	7	1078	December 22, 2025
Crawl specific page for the answer General Questions ai-chatbot	5	27	February 9, 2026
AI Chatbot: [MAJOR UPDATE] Sitemap training, new language model and more! 🚀 New Feature ai-chatbot	11	577	January 19, 2026

URL-based data scraping to create a knowledge base

Related topics