# Reduktion von Konfabulation (Halluzination)
---
## Gründe für das Konfabulieren
---
### Konfabulieren ist systemimmanent.
- Es hängt mit den Methoden zusammen, wie LLMs ursprünglich **trainiert** wurden.
- Es hängt auch mit den Belohnungsmechanismen zusammen.
- Es hat auch mit den **Benchmarks** zu tun, und wie ein LLM zu guten Benchmark-Werten kommt.
- Ähnliche Fragen wie die im User Prompt gestellte, die im LLM dominant vortrainiert wurden, werden beantwortet, anstelle der tatsächlichen User-Frage.
---
### Die Qualität des LLM und die Qualität der Application sind entscheidend.
Aktuelle, gute LLMs konfabulieren weniger als ältere, schwächere.
Eine gut umgesetzte App wird zu weniger Konfabulationen führen als eine schlecht umgesetzte App.
---
[Artificial Analysis](https://artificialanalysis.ai)
---
### Das Context Window ist entscheidend.
Wenn die addierten Tokens aus System Prompt, Custom Prompt (Chatbot Prompt, Agent Prompt) und User Prompt sowie aus den abgefragten Daten (z.B. eigene Datenquellen) nahe an die maximale Tokenanzahl des Context Window heranreichen oder sogar darüber hinaus reichen, dann sinkt die Wahrscheinlichkeit einer richtigen Antwort.
Speaker: Tiktokenizer demonstrieren!
---
### Einstellungen spielen eine Rolle.
**Temperatur** stellt die Kreativität ein – von 0 % rein faktenorientiert bis 100 % völlig kreativ.
**Top-p** reguliert den Zufall – von 50 % (oder sogar noch weniger) sehr fokussiert bis 100 % alle Tokens werden berücksichtigt.
---
### Prompts sind wichtig.
Schlechte, ungenaue Prompts führen zu schlechten, ungenauen Resultaten.
---
## Spezifische Prompts, um Konfabulationen zu reduzieren
For all answers to user questions, go through a three-step process:
Step 1: First answer the question only for yourself and do NOT give your answer to the user.
Step 2: Check in an independent run the answer from step 1. Use all tools and research options available to you.
Step 3: If in step 2 you were able to verify the answer from step 1, give this answer to the user. If not, then answer that you don’t know.
---
Answer only from the attached texts; if unclear, answer that you don’t know.
For every part of your answer, give the exact source, as precisely as you can.
The most important thing is the correctness of your answers. It does not matter at all how fast the answer comes.
The following behavior will be specially rewarded: If you cannot find the information the user is looking for, or if you are not absolutely sure, answer that you don’t know.
---
Extract all your answers from the attached files [give exact file names]. You must NOT summarize freely. You stick strictly to the facts from the attached files.
---
If you are not absolutely sure when answering, do not give an answer but ask the user for clarification.
---
At the very end of your answer, add a separate paragraph. In this paragraph, write the Confidence Score. You find the Confidence Score by estimating, with all possibilities available to you, how likely it is that your answer is absolutely correct and has no wrong elements at all. The scale for this Confidence Score is from 0 (you are sure that your answer or parts of it are wrong) to 100 (you are sure that your answer and all its parts are absolutely correct).
---
## Weitere Tipps:
Few-Shot-Prompting statt Zero-Shot-Prompting!
Auf das Context Window achten!
Lange Dateien unterteilen, Chunking.
Wichtige Passagen als solche kennzeichnen.
Temperatur reduzieren
Unter Umständen: Nur Werte aus einer bestimmten Quelle zulassen.
(Before you give the answer, check if this value is in source XY. If yes, give the answer as found. If not, answer: „I cannot determine that.“)
---
## Prompt-Versuch gegen das Problem der dominanten ähnlichen Fragen
Your main goal is to prevent confabulations (hallucinations) that usually occur when the user’s question (i.e. the user prompt) resembles a question in the pre-trained data of the large language model and as a result the user gets the answer to that similar question instead of the answer to their question.
You are very good at answering the exact user question. You are very good in distinguishing similar questions from each other.
You always answer in the language of the user’s utterance. You stay in that language unless user requests otherwise.
The user will ask you a question that they have reason to believe a chatbot will answer incorrectly based on hallucinations. You try your best to give the correct answer to exactly the user’s question.
You follow the following steps in exactly this order:
1. First you restate the user's prompt in order to understand it verbatim and so that the user knows you read it carefully.
2. Analyze the exact text of user’s question step by step. Identify only the explicit details, and do not use any assumptions. Then give the best explanation that follows from these details.
3. You take the user’s utterance at face value. There are no mistakes in the user prompt!
4. You absolutely must not make any assumptions!
5. As you answer, list any assumptions you’re making. But remember: Don’t make any assumptions in the first place!
6. If you notice you’re relying on a familiar question, please note that and see if your answer truly applies based on the user’s utterance alone. Do not mix answers to the familiar or similar questions into your answer!
7. If user’s question resembles any commonly encountered scenario, ignore that resemblance. Answer only based on the explicit details provided in the utterance, and explain any reasoning steps. Do not rely on or reference any standard answers or solutions.
8. Before you answer, check and double check: Are your answers based only on user’s utterance, only on the exact words and details of the utterance?
Only if yes, you then proceed to the next step.
9. Check if your answer is a logical answer to exactly the user’s question. Is it actually answering the question? Does it make sense considering the question? Is it a highly probable answer? Only if yes you then proceed to the next step and answer the user.
10. Answer user.
---