Monitor BetterDocs AI Chatbot Performance in Observability

BetterDocs AI Chatbot helps your website’s visitors get instant answers from your documentation. With the Observability, you can see how well the chatbot is doing: how many chats it handles, how fast it replies, how much it costs, and which questions your docs cannot answer yet. In this documentation, we will show you how to monitor BetterDocs AI Chatbot performance with Observability.

ध्यान दें: Before getting started, you need to install and activate the BetterDocs AI Chatbot addon to proceed.

Step 1: Navigate to BetterDocs AI Chatbot Observability #

From your WordPress dashboard, navigate to ‘BetterDocs’ → ‘AI Chatbot’. Then click on the ‘Observability’ tab from the left panel.

You will find 4 tabs here: ‘Overview’, ‘Requests’, ‘Models & cost’ तथा ‘Retrieval’. The same toolbar sits on top of every tab:

Time range: Pick ‘1 h’, ’24 h’, ‘7 d’ या ’30 d’ to choose the period you want to check.
Audience filter: Use the dropdown (set to ‘Visitors’ by default) to choose whose chats you want to see.
Refresh: Hit the ‘Refresh’ button to load the latest data instantly. The data also refreshes on its own every 30 seconds.
Settings: Click ‘Settings’ to adjust the Observability options.

Step 2: Check the Chatbot Health from the Overview Tab #

The ‘Overview’ tab gives you a quick summary of how your AI Chatbot is doing in the selected period. At the top, you will see 6 metric cards:

Metric	What It Shows
Chat turns	Total number of questions the chatbot answered, compared with the previous period.
Turn rate	Average chat turns per hour, along with the peak turns per minute.
p95 latency	Time the chatbot takes to reply for 95% of chats, compared with the target (3.00 s) and the p50 (median) time.
Error rate	Percentage of chat turns that failed, along with the number of failed turns.
Tokens	Total tokens used, split into input (In) and output (Out) tokens.
Estimated spend	Approximate AI cost for the selected period, calculated at list prices.

ध्यान दें: If a card is highlighted in orange, that metric is outside its target. For example, a p95 latency above 3.00 s means visitors are waiting longer than expected for answers.

Below the cards, you will find:

Chat turns: A chart that shows how many chats came in over time.
End-to-end latency: A chart that shows the time from a visitor’s message to the answer, with p50, p95 and the 3.00 s target line.
Pipeline stage health: The p95 time for each step of a chat turn, so you can find which step slows it down. This section needs an AI service version with per-turn traces.
Recent turns: A list of the latest chats with their time, trace ID, intent, latency, tokens and outcome. Click ‘All requests’ to see the full list in the ‘Requests’ टैब।

Monitor BetterDocs AI Chatbot Performance

Step 3: Review Every Chat Turn from the Requests Tab #

के पास जाओ ‘Requests’ tab to see every chat turn in a detailed list. Each row shows the Time, Trace, Intent, Model, Latency, Tokens, Cost तथा Outcome of the turn.

The Outcome column tells you how the chatbot handled each question:

Answered: The chatbot found the answer in your docs and replied.
No match: The chatbot could not find anything in your docs to answer from.
Human Support: The chat was handed over to your support team.
Asked to clarify: The chatbot asked the visitor for more details.

To find specific chats quickly, use the filters on top:

Search: Enter a trace or session ID to find a particular chat.
Outcome, Intent & Model: Pick an option from each dropdown to narrow down the list.
Slower than 3.00 s: Tick this checkbox to see only the slow replies.

You can also hit the ‘Export CSV’ button to download the list and analyze it later. Use the ‘Previous’ तथा ‘Next’ buttons to move between pages.

Step 4: Track AI Costs from the Models & Cost Tab #

Switch to the ‘Models & cost’ tab to keep your AI spending under control. Here, you will see 4 cost cards:

Estimated spend: The total AI cost for the selected period.
30-day projection: What you may spend in 30 days if usage continues at the current daily pace.
Cost per turn: The average cost of each priced chat turn.
Answered from cache: The percentage of repeated questions answered from cache. These answers skip the AI calls, so they cost nothing extra.

Below the cards, the ‘By chat model’ table breaks down the usage for each AI model (for example, gpt-4.1). It shows the Turns, Input tokens, Output tokens, Avg latency तथा Estimated cost for each model. This helps you compare models and pick the one that gives the best balance of speed and cost.

Step 5: Find Content Gaps from the Retrieval Tab #

The ‘Retrieval’ tab shows how well your docs cover the questions visitors ask. At the top, you will find 3 cards:

No-match rate: The percentage of chat turns where the chatbot found nothing in your docs to answer from.
Avg best score: The average cosine similarity of the closest doc chunk. A higher score means your docs match the questions better.
Chunks per answer: The average number of doc passages the chatbot used to build each answer.

Below the cards, the ‘No-match rate over time’ chart shows how your no-match rate changes. The ‘Best match score’ chart groups chat by their closest doc chunk. Bars under 0.3 are questions your docs barely cover.

At the bottom, the ‘Recent questions the bot couldn’t answer’ section lists the exact questions that got no match, with the date and time. Use this list to write new docs or update existing ones, so the chatbot can answer these questions next time.