AI Chatbot Not Respecting Pre-Set "Quick Reply" Answers

[Petar_Dietrich]

Thanks, @NRV_Wayfinder,

I will try your suggestion (i.e., convert Q&As to PDF format and upload the file) and report back if there are any improvements in the chatbot’s answers.

So you know, I’ve updated our internal chatbot training instructions (almost “iron-clad”) and the chatbot’s answers are much better.

Again, thank you!

[Petar_Dietrich]

Hey @Max,

Given all the different AI Chatbot training fields available in the configurator, why doesn’t Elfsight simply define a “hierarchy” directive for each configurator training field? Is that currently defined in your LLM code? If so, what is the hierarchy?

For example, our AI Chatbot Training Instructions contain the below directives (only partial shown), but it’s hit or miss.

Can you look into that?

Thank you!

---

## SECTION 1 — Q&A OVERRIDE (HIGHEST OPERATIONAL PRIORITY)
If a user query matches internal knowledge base content (Q&A module):
- **Role Change:** You are a **Copy-Paste Engine**, not a conversational writer.
- **Action:** Respond **verbatim**. Output the text exactly as it appears in the source, character-for-character.
- **Constraints:** Do **not** paraphrase. Do **not** condense or expand. Do **not** add "Nova" branding, greetings, or closings.
- **Formatting:** Ignore Section 6 (Structure) and Section 14 (Length) for Q&A matches. Provide the raw text only.
## SECTION 5 — RESPONSE PRIORITY HIERARCHY
1. Security & Compliance (Section 0)
2. **Q&A Override (Section 1)**
3. Knowledge Sourcing
4. Date Qualifier (if external)
5. Lead Capture (if applicable)
6. Formatting & Brand Standards

[Max]

Hey there, @Petar_Dietrich

Based on the instruction snippet you’ve shared, in section 1 you’re trying to make the bot use the exacr answers from the Q&A when users query match the questions from the Q&A section.

As mentioned in the previous messages, you can make the bot do this given your instructions is concise and not overloaded.

It seems you’ve edited the instruction and it became even longer and more complex than it was, which makes it hard for the current LLM to handle all the directives from your instruction.

I heop this clarifies things.

Regarding the hierarchy, would you like to know if the training options (Training Files, Q&A, Text blocks, Instruction) have different priority level written in the internal instruction?

[Petar_Dietrich]

Howdy, @Max,

Our current AI Chatbot Training Instructions suit us well for now and cover 80% of our issues. So you know, our instructions are still unable to get your chatbot to give priority to our Q&A section. Others have confirmed this with you as well (as noted in the community forum).

The answer to your question is Yes! I would simply like to know if your developers have assigned – per their internal instructions – a “hierarchy” directive to the many training fields available in the configurator. See screenshot below. If not, it would help a lot!

Thank you!

---

[Petar_Dietrich]

Hi Team!

I have my answer. Courtesy of OpenAI (ChatGPT) and our company testing.

Elsight AI Chatbot Configurator Processing Hierarchy:

1. Agent Instructions
2. Questions & Answers
3. Text Blocks
4. Files
5. Web Pages

Elsight AI Chatbot Q&A Processing and Web Search Capabilities: (see attached)

For the ChatGPT-5 mini model (used by Elfsight):

1. There is no condition or instruction available that guarantees the chatbot will provide the exact same answers contained in the Questions and Answers section. The chatbot processes them and provides a similar answer based on internal LLM logic.

2. The chatbot’s web search capabilities are limited due to its native “training” cutoff period and native architecture designed for speed and cost. The chatbot may answer queries that require external sources for its answers, but it’s not guaranteed the chatbot will always “find” an answer. Note: GPT‑5 mini can perform a full web search if the developers of Elfsight enable it (via RAG/API integration), but doing so incurs additional costs beyond standard model usage.

Elfsight AI Chatbot Configurator Processing Hierarchy (2026-02-16).pdf (233.7 KB)

Understanding ChatGPT 5 Mini - Web Search and Knowledge Limitations (2026-03-16).pdf (4.0 MB)

[Max]

Hi Petar

Thanks so much for taking the time to look into these questions.

Our developers have reviewed your findings, and here’s what they’ve shared:

1. Web Search
   You’re absolutely right! Due to the training cutoff of the current LLM and the costs associated with this feature, a global web search is not supported at the moment.

2. Q&A Processing
   The info you’ve found isn’t entirely accurate.

The agent doesn’t distinguish between Q&A, Web Pages, Text Blocks, or Files, so there’s no hierarchy among these training methods, but the AI Instructions always take priority over them.

When a user asks a question, a semantic (vector-based) search is performed. The search results are split into chunks and inserted into the response context without any ranking. OpenAI also provides details on which questions were used for the search and which files contributed to the answer.

Here’s a screenshot from their internal tool for reference.

[Petar_Dietrich]

Hi Max,

Thank you so much for the follow-up.

Concerning the developers statement:

When a user asks a question, a semantic (vector-based) search is performed.

Is the vector-based search based on the natural behavior of the LLM (ChatGPT-5 Mini) or your developers’ internal directives/programming for the LLM?

Again, thank you.

[Max]

Natural behavior of the LLM