Passing a loaded client to dspy.LM instead of using API calls.

[kgourgou]

Hi all, quick question about dspy.LM.

I'm trying to migrate to using dspy.LM, but I've hit a snag. From the migration notebook and the guide here, I see that dspy.LM can work with clients that are behind some API endpoint, either local or hosted somewhere else.

What is not clear to me is how to pass an already instantiated OpenAI client to dspy.LM. In theory it should be simple, but I noticed that litellm wants to load / query the model from a URL.

Imagine the following:

# this function loads an  Azure OpenAI client by using additional checks, e.g., certificate files, usernames, etc., in addition to the usual API key and various URLS. 
openai_model = custom_made_loading_function(api_key, certificates, usernames, etc)

# what I would like to do 
model = dspy.LM('openai/gpt4', client=openai_model)

# then continue using dspy as usual :-) 

My understanding is that I cannot do what I want currently because litellm is a necessary call within dspy.LM.call() and litellm will try to figure out how to authenticate and send a request to the API.

Is this correct? Do you think there is a way around?

[okhat]

Hey, @kgourgou ! If you can do it via litellm, you should be able to do it via dspy.LM, since it's a shallow wrapper.

[kgourgou]

I'll take a look!