A Generative AI Engineer received the following business requirements for an external chatbot.
The chatbot needs to know what types of questions the user asks and routes to appropriate models to answer the questions. For example, the user might ask about upcoming event details. Another user might ask about purchasing tickets for a particular event.
What is an ideal workflow for such a chatbot?

• A. The chatbot should only look at previous event information
• B. There should be two different chatbots handling different types of user queries.
• C. The chatbot should only process payments
• D. The chatbot should be implemented as a multi-step LLM workflow. First, identify the type of question asked, then route the question to the appropriate model. If it's an upcoming event question, send the query to a text-to-SQL model. If it's about ticket purchasing, the customer should be redirected to a payment platform.

A generative AI engineer is deploying an AI agent authored with MLflow's ChatAgent interface for a retail company ' s customer support system on Databricks. The agent must handle thousands of inquiries daily, and the engineer needs to track its performance and quality in real-time to ensure it meets service-level agreements. Which metrics are automatically captured by default and made available for monitoring when the agent is deployed using the Mosaic AI Agent Framework?

• A. Both operational and quality metrics
• B. Quality metrics like correctness and guideline adherence
• C. Operational metrics like request volume, latency, and errors
• D. No metrics are automatically captured

A Generative Al Engineer is setting up a Databricks Vector Search that will lookup news articles by topic within 10 days of the date specified An example query might be " Tell me about monster truck news around January 5th 1992 " . They want to do this with the least amount of effort.
How can they set up their Vector Search index to support this use case?

• A. pass the query directly to the vector search index and return the best articles.
• B. Split articles by 10 day blocks and return the block closest to the query.
• C. Create separate indexes by topic and add a classifier model to appropriately pick the best index.
• D. Include metadata columns for article date and topic to support metadata filtering.

A Generative Al Engineer has created a RAG application to look up answers to questions about a series of fantasy novels that are being asked on the author's web forum. The fantasy novel texts are chunked and embedded into a vector store with metadata (page number, chapter number, book title), retrieved with the user' s query, and provided to an LLM for response generation. The Generative AI Engineer used their intuition to pick the chunking strategy and associated configurations but now wants to more methodically choose the best values.
Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and parameters? (Choose two.)

• A. Change embedding models and compare performance.
• B. Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most appropriate chunk. Optimize the chunking parameters based upon the values of the metric.
• C. Pass known questions and best answers to an LLM and instruct the LLM to provide the best token count. Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
• D. Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the chunking strategy, such as splitting chunks by paragraphs or chapters.
  Choose the strategy that gives the best performance metric.
• E. Add a classifier for user queries that predicts which book will best contain the answer. Use this to filter retrieval.

A Generative AI Engineer is tasked with deploying an application that takes advantage of a custom MLflow Pyfunc model to return some interim results.
How should they configure the endpoint to pass the secrets and credentials?

• A. Use spark.conf.set ()
• B. Pass the secrets in plain text
• C. Pass variables using the Databricks Feature Store API
• D. Add credentials using environment variables