AI Chatbot: The next step forward with GPT-6 Luna [update rolled back]

Tips & Discussions Community Discussions
1

We’ve upgraded AI Chatbot once again — it now runs on the even more advanced GPT-6 Luna model! :rocket:

Following our recent switch to GPT-5.6 Luna, we’ve taken another step forward to make conversations with the chatbot even smoother, more accurate, and better at understanding your requests.

Try it out and let us know how the updated experience feels on your end :blush:


Back to Changelog →

2

Hello @Max! :sun:

Thanks for the update! Our team will keep an eye on the chatbot’s performance under GPT-6-luna and let you know if we spot any issues.

Also, to easily confirm that a customer’s chatbot was successfully switched, would it be possible for your developers to add an LLM indicator to the AI Agent Profile dashboard? See the below image for the optimum location and formatting framework. Having the indicator update automatically whenever the LLM is changed would be a perfect way to confirm everything is working and close the loop.

Thanks again!

ai-chatbot-widget-llm-version-indicator
ai-chatbot-widget-llm-version-indicator1203×668 30.1 KB

3

Hi Petar,

Thanks a bunch for the feedback! :raising_hands:

I remember you previously suggested this improvement, and we’ve already added it to our Wishlist. I’ve passed your feedback along to the team again, so they can keep it in mind as we continue improving the AI Agent dashboard.

If there are any updates on this request, we’ll share them directly in the Wishlist thread :slightly_smiling_face:

4

Max, I don’t know if others are experiencing the same thing, but since the switch to GPT-6 Luna, the chatbot’s performance has dropped noticeably. There are many questions I ask that it used to answer well but now says it cannot answer. Is anyone else experiencing this? The responses were much better with GPT-5.6 Luna.

5

Hi there, @Gavi_Hard_Soft :waving_hand:

Thank you so much for bringing this to our attention!

I’ve passed your feedback to the devs and will get back to you once I have any news :slightly_smiling_face:

6

@Gavi_Hard_Soft, could you please share:

  • the questions your bot stopped answering correctly

  • the expected answers to these question

This info would help us investigate the issue :folded_hands:

7

Hi Max. Last night, a few hours after sending the message, I tried the chatbot again and it worked fine. It might have been a temporary issue earlier on. Specifically, it was responding to most simple and common questions with “I can’t answer that right now,” but a few hours later, it resolved itself. Thanks for your reply. Everything is fine for the moment.

8

Happy to hear it’s working smoothly now! Thanks for the update :wink:

9

All,

You can view the reasoning-vs.-cost-efficiency trade-off between GPT-5.6-luna and GPT-6-luna here:

In short, given that GPT-6-luna is cheaper to operate, the answers are not as good. Our team and customers noticed this during testing. To compensate for this, we now have to rely on Conversation Rules alone rather than a combination of Conversation Rules, Text Blocks, and Q&As in the training module.

This begs the question: What type of evaluation and testing is Elfsight conducting to ensure that the quality of its chatbot’s answers does not suffer when it switches to a cheaper OpenAI model, potentially at the customer’s expense? What decision-making logic does Elfsight follow when selecting which model to use?

Also, since customers are paying for the previously used, “smarter,” and more expensive OpenAI models, is Elfsight going to reduce the monthly price of its AI Chatbot widget accordingly? What compensating measures, if any, will Elfsight offer its customers?

Last, please keep this in mind: A newer, higher OpenAI model version does not always mean it’s more advanced. In this case, it’s cheaper but not necessarily better.

Thank you.

10

Hello Petar,

Thanks for the detailed feedback and for taking the time to share your observations!

I’m sorry that the model change has impacted the quality of your chatbot responses. We understand how important response accuracy and consistency are, and this is something we’re taking seriously.

I’ve shared your feedback and questions with our devs so they can review the situation.

I’ll keep you updated once I have any news from the team :folded_hands:

16

Hello Petar,

We’ve reviewed the feedback we received after moving AI Chatbot from GPT-5.6 Luna to GPT-6 Luna, and we’ve decided to switch back to GPT-5.6 Luna.

Before the rollout, we compared both models across a number of test cases and didn’t see a meaningful difference in overall answer quality. However, after the release, real customer conversations showed issues that our initial testing didn’t capture well enough.

Going forward, we’ll put more weight on real AI Chatbot use cases when evaluating model changes, rather than relying mainly on general benchmarks or pre-release testing. A newer model version also won’t automatically mean it’s the better choice for our product.

To answer your question about cost: this change was not primarily made to reduce the cost of running AI Chatbot. Our main priority is the quality and reliability of the experience. Based on the feedback we’ve seen, GPT-5.6 Luna is currently the better fit, which is why we’re reverting the change.

Thank you again for bringing this to our attention and sharing your detailed feedback, we really appreciate it :slightly_smiling_face:

17

Hi @Max,

Thank you. Smart move!

From my research and testing, GPT-5.6 Luna is currently the most cost-effective, reasoning-effective OpenAI language model for Elfsight’s AI chatbot.

Cheers!

18

Thanks, Petar! Glad to hear that :blush:

We really appreciate you taking the time to test the models and share your findings — it’s very helpful to have that perspective alongside what we’re seeing from real customer conversations.