Chat with your Snowflake Data and LLMs

Snowflake provides a great Quickstart guide on how to set up a RAG based LLM assistant via Snowflake Cortex (shoutout to Carlos Carrero). And please don't get me wrong, using Streamlit as the interface for the conversation with end users is a perfectly fine approach. However, I would like to have the same thing, but inside MS Teams 😎

(side note: not because I am particularly fond of Teams, but it's widely used in my org, so I don't really want to teach anyone where to find yet another tool)

But first, here is the Quickstart on using your own data in Snowflake Cortex:

quickstarts.snowflake.com/guide/asking_questions_to_your_own_documents_with_snowflake_cortex

Now, how do I get this into Teams?

Chatbot in Teams

Sure, you could use MS Copilot Studio to create a Teams chatbot and there are lots of other ways. However, you could also get a (rather affordable) Power Automate Premium license (or Azure Logic Apps if you prefer) and give it to a service user you already have in Teams.

Power Automate is capable of receiving direct chat messages (or, if you prefer, channel messages, too) in Teams as a trigger. Since you only want it to react to messages written into the chat by someone else but the (service-) user running the flow, you need to add a little filter:

{
  "not": {
    "equals": [
      "@body('Get_message_details')?['from']?['user']?['id']",
      "@variables('user_id')"
    ]
  }
}

Sending the message to Snowflake

There is another Quickstart guide on how to use Power Automate and Snowflake:

quickstarts.snowflake.com/guide/power_apps_snowflake

Using the "Submit SQL Statement for Execution" action you can send a prompt containing the message received from Teams over to Snowflake Cortex as a query. Here, I simply send over the plain message received to mistral-7b (pick whichever Snowflake LLM you like):

select SNOWFLAKE.CORTEX.COMPLETE(
  'mistral-7b'
  , '@{body('Get_message_details')?['body']?['plainTextContent']}'
) as response

Whatever the COMPLETE function returns will then be received by Power Automate again and can be pushed to a new Teams message in the same chat triggering the flow in the first place:

And with that, you can chat with Snowflake LLMs:

Considerations

This simple approach uses a very simple prompt without any context. Obviously, the whole point of setting this up, is to enable you to chat with your own data (aka RAG based LLM) in Snowflake. That requires a more complex prompt, but the first Quickstart on the topic will lead you there 😉

Also, this simple Power Automate Flow does not consider any chat history. If you want that, the flow should be extended to add previous messages in the chat to the context and the prompt. But I'll leave this open for someone else to talk about 😅