URL-based data scraping to create a knowledge base for AI Assistant

We’d like to provide a website URL, and the assistant will scrape the content to create knowledge base.

15 Likes

This would be great. I was also proposing to be able to provide the URL to a sitemap, so that the assistant could scrape the content of all the URLs listed in the sitemap:

Training AI Chat Bot with sitemap

6 Likes

It would be great if this could be done dynamically when initializing the bot. In our case we have a car rental website and we get a lot of questions regarding the terms and conditions, insurance included, etc. This differs from car to car, so we are not able to load the chatbot with just one single file explaining this.

It would be great if at initialization we could load the chatbot with the terms and conditions for the car the user is looking at, so any answer is specific to that car.

Cheers,

2 Likes

Hi there and welcome to the Community, @user18897 :wave:

Thank you so much for sharing your use case with us! We’ll have it in mind when considering this idea :slightly_smiling_face:

2 Likes

Thank you for this tool. Wouldn’t it be possible for the tool to read the content directly from the website instead of having to manually upload all the FAQs, for example?

4 Likes

Hi @Matteo :wave:

Right now, it’s impossible to do, but we already have a similar idea on the Wishlist. I’ve moved your comment to the related thread, where we’ll keep you in the loop :slightly_smiling_face:

3 Likes

I find it difficult to maintain content by posting articles in the configuration tool.

A periodic scrape of our website, or even a button I could press in the configuration tool to tell it that new content has arrived would be fantastic.

Or an API for content management so we could automate?

Thank you for this fantastic widget!

5 Likes

HI @Loren_West :wave:

Thank you so much for your feedback and suggestions! We’ll have them in mind :wink:

This would be wonderful. Right now I’m just copy/pasting websites into a document and uploading them, which is labor intensive, and kind of silly.

1 Like

Thanks for the email, Can you please show me how to do it?

This would be great if it could be done live and dynamically! Include RSS, JSON, XML, etc, along with web scraping.

2 Likes

We’d like to provide a website URL, and the assistant will scrape the content to create a knowledge base. This would be great if it could be done live and dynamically! Include RSS, JSON, XML, etc., along with web scraping.

1 Like

The chat needs to take a deeper look into the site and understand the site’s features, subscriptions and more.

1 Like

Thanks a lot for your comments and suggestions, guys! I hope we’ll be able to make our AI Chatbot smarter in future updates. I’ll be happy to update you here if there’s any news or progress :slight_smile:

1 Like

I have a similar request to this in another thread! Having the ability to dynamically read an external file such ass rss, json, atom, etc. Perhaps it could still work with PDF’s and Docs, CSV files..

This would be awesome for sure! I have a separate request for that myself would love to get some votes! Or perhaps it can be included in this request!

It seems like there are a lot of request in here for this to be live & dynamic! I created a separate request for that feature not realizing how many people have also requested it. I’d love to see this happen!

How do I vote this up x10? :slight_smile:

1 Like

We love this idea too, @B_F! Let’s hope we’ll be able to include it in one of the next releases :blush:

1 Like