Dynamic Routing¶

Bridgic supports dynamic routing to the corresponding worker based on runtime conditions. Now let's understand it with a sample example.

Domain chatbot¶

Dynamic routing is a common requirement in chatbots. For instance, it involves analyzing the input query to determine the main domain or field it pertains to, and then dynamically routing it to the corresponding processing logic.

The user inputs the original query, we identify whether it is a medical-related or legal-related issue, and then we answer it with specialized logic. There are three steps:

Receive user input
Domain recognition and dynamic routing to correct worker
Answer query

1. Initialize¶

Before we start, let's prepare the running environment.

In [ ]:

  Copied!     
 
# Get the environment variables.
import os
from typing import Literal

_api_key = os.environ.get("OPENAI_API_KEY")
_api_base = os.environ.get("OPENAI_API_BASE")
_model_name = os.environ.get("OPENAI_MODEL_NAME")

# Import the necessary packages.
from pydantic import BaseModel
from bridgic.core.model.protocols import PydanticModel
from bridgic.core.automa import GraphAutoma, worker
from bridgic.core.model.protocols import Choice
from bridgic.core.model.types import Message, Role
from bridgic.llms.openai import OpenAILlm, OpenAIConfiguration

llm = OpenAILlm(  # the llm instance
    api_base=_api_base,
    api_key=_api_key,
    configuration=OpenAIConfiguration(model=_model_name),
    timeout=20,
)
# Get the environment variables. import os from typing import Literal _api_key = os.environ.get("OPENAI_API_KEY") _api_base = os.environ.get("OPENAI_API_BASE") _model_name = os.environ.get("OPENAI_MODEL_NAME") # Import the necessary packages. from pydantic import BaseModel from bridgic.core.model.protocols import PydanticModel from bridgic.core.automa import GraphAutoma, worker from bridgic.core.model.protocols import Choice from bridgic.core.model.types import Message, Role from bridgic.llms.openai import OpenAILlm, OpenAIConfiguration llm = OpenAILlm( # the llm instance api_base=_api_base, api_key=_api_key, configuration=OpenAIConfiguration(model=_model_name), timeout=20, )

2. Complete¶

We assume that the user query we receive is a string. Let's implement the domain chatbot to answer the user query.

In [ ]:

  Copied!     
 
class Domain(BaseModel):
    choices: Literal["medical", "legal"]
class Domain(BaseModel): choices: Literal["medical", "legal"]

In [ ]:

  Copied!     
 
class DomainChatbot(GraphAutoma):
    @worker(is_start=True)
    async def pre_query(self, query: str):  # receive user input and preprocess it
        return query

    @worker(dependencies=["pre_query"])
    async def domain_recognition(self, query: str):  # domain recognition and dynamic routing
        response: Domain = await llm.astructured_output(
            constraint=PydanticModel(model=Domain),
            messages=[
                Message.from_text(text="Please identify the domain of the following query", role=Role.SYSTEM),
                Message.from_text(text=query, role=Role.USER),
            ]
        )
        print(f'The query domain is: {response}')  # print the response
        
        # dynamic routing
        if response.choices == "medical":
            return self.ferry_to('answer_medical_query', query)
        elif response.choices == "legal":
            return self.ferry_to('answer_legal_query', query)
    
    @worker(is_output=True)
    async def answer_medical_query(self, query: str):  # answer medical-related query
        response = await llm.achat(
            messages=[
                Message.from_text(text="You are a medical expert. Answer the following medical-related query", role=Role.SYSTEM),
                Message.from_text(text=query, role=Role.USER),
            ]
        )
        return response.message.content

    @worker(is_output=True)
    async def answer_legal_query(self, query: str):  # answer legal-related query
        response = await llm.achat(
            messages=[
                Message.from_text(text="You are a legal expert. Answer the following legal-related query", role=Role.SYSTEM),
                Message.from_text(text=query, role=Role.USER),
            ]
        )
        return response.message.content
class DomainChatbot(GraphAutoma): @worker(is_start=True) async def pre_query(self, query: str): # receive user input and preprocess it return query @worker(dependencies=["pre_query"]) async def domain_recognition(self, query: str): # domain recognition and dynamic routing response: Domain = await llm.astructured_output( constraint=PydanticModel(model=Domain), messages=[ Message.from_text(text="Please identify the domain of the following query", role=Role.SYSTEM), Message.from_text(text=query, role=Role.USER), ] ) print(f'The query domain is: {response}') # print the response # dynamic routing if response.choices == "medical": return self.ferry_to('answer_medical_query', query) elif response.choices == "legal": return self.ferry_to('answer_legal_query', query) @worker(is_output=True) async def answer_medical_query(self, query: str): # answer medical-related query response = await llm.achat( messages=[ Message.from_text(text="You are a medical expert. Answer the following medical-related query", role=Role.SYSTEM), Message.from_text(text=query, role=Role.USER), ] ) return response.message.content @worker(is_output=True) async def answer_legal_query(self, query: str): # answer legal-related query response = await llm.achat( messages=[ Message.from_text(text="You are a legal expert. Answer the following legal-related query", role=Role.SYSTEM), Message.from_text(text=query, role=Role.USER), ] ) return response.message.content 

Let's run it!

In [5]:

  Copied!     
 
query = "What is the aspirin?"
domain_chatbot = DomainChatbot()
await domain_chatbot.arun(query)
query = "What is the aspirin?" domain_chatbot = DomainChatbot() await domain_chatbot.arun(query)

The query domain is: medical

Out[5]:

'Aspirin, also known as acetylsalicylic acid, is a widely used medication with several important medical properties. It is one of the oldest and most well-known over-the-counter (OTC) drugs, originally derived from willow bark.\n\n### Key Uses of Aspirin:\n\n1. **Pain Relief (Analgesic)**  \n   Aspirin helps relieve mild to moderate pain such as headaches, toothaches, muscle aches, and menstrual cramps.\n\n2. **Reducing Fever (Antipyretic)**  \n   It lowers body temperature in cases of fever, such as in infections.\n\n3. **Reducing Inflammation (Anti-inflammatory)**  \n   Aspirin helps reduce inflammation, which can be beneficial in conditions like rheumatoid arthritis or other inflammatory disorders.\n\n4. **Preventing Blood Clots (Antiplatelet Effect)**  \n   This is one of its most important therapeutic uses. Aspirin inhibits the formation of blood clots by blocking the action of a substance called cyclooxygenase (COX), which reduces the production of thromboxane—a molecule that promotes platelet aggregation.  \n   - **Low-dose aspirin (typically 81 mg)** is commonly prescribed for long-term use to prevent heart attacks and strokes in individuals at risk due to cardiovascular disease (e.g., history of heart attack, angina, or stroke).\n\n5. **Cardiovascular Protection**  \n   Regular low-dose aspirin is recommended for secondary prevention in patients with a history of cardiovascular events to reduce the risk of recurrence.\n\n---\n\n### How It Works:\nAspirin irreversibly inhibits the enzyme COX-1 and COX-2, which are involved in producing prostaglandins. Prostaglandins play roles in inflammation, pain, fever, and platelet aggregation. By blocking COX-1, aspirin reduces platelet activation and clot formation.\n\n---\n\n### Safety and Precautions:\n- **Gastrointestinal Side Effects**: Aspirin can irritate the stomach lining and increase the risk of ulcers or bleeding, especially with long-term or high-dose use.\n- **Bleeding Risk**: Due to its antiplatelet effect, it increases the risk of bleeding (e.g., gastrointestinal bleeding, bruising).\n- **Allergic Reactions**: Some people may have a severe allergic reaction, including asthma exacerbation (especially in those with a history of aspirin sensitivity).\n- **Reproductive Health**: Aspirin is not recommended during pregnancy, especially in the third trimester, due to risks of fetal complications.\n- **Children and Fever**: Aspirin should be avoided in children and adolescents with viral infections (like chickenpox or flu) due to the risk of Reye’s syndrome, a rare but serious condition.\n\n---\n\n### Summary:\n**Aspirin is a versatile medication used for pain, fever, inflammation, and cardiovascular protection. Its most significant medical role today is in preventing heart attacks and strokes through long-term low-dose use. However, it should be used cautiously and only as directed by a healthcare provider.**\n\nAlways consult a doctor before starting or stopping aspirin therapy, especially if you have underlying health conditions or are taking other medications.'

If you encounter a Readtimeout error during the execution of the above process, it might be because the timeout period set during the initialization of the llm is too short, causing the model to time out before it finishes responding.

Great! We have successfully completed the domain chatbot.

What have we done?¶

We have implemented the routing mechanism using Bridgic.

Ferry To¶

In Bridgic, we use the ferry_to mechanism to achieve dynamic routing. Its first parameter is the key of the worker to which the route will be directed, and the second parameter is the input parameter passed to the target worker.