Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the way of enable/disable litellm logging #1777

Merged
merged 1 commit into from
Nov 9, 2024

Conversation

chenmoneygithub
Copy link
Collaborator

We introduced litellm logging suppression in #1768, but we were configuring litellm inside the __init__ call. So if users do:

lm = dspy.LM("...")
dspy.settings.configure(suppress_debug_info=False)
lm("some query")

Then the logging won't be turned on. In this PR I am introducing a pair of util functions: dspy.clients.enable_litellm_logging() and dspy.client.disable_litellm_logging() to turn on/off the litellm logging dynamically.

@okhat
Copy link
Collaborator

okhat commented Nov 8, 2024

Thank you, @chenmoneygithub! I like this! But should it just be dspy.configure(litellm_logging=True/False)?

If you think the answer is no, feel free to merge this!

@chenmoneygithub
Copy link
Collaborator Author

chenmoneygithub commented Nov 9, 2024

@okhat Thanks! I took a second look at dspy.settings.configure(), and seems right now it's just updating the global state dict instead of running any dynamic configuration:

    def configure(self, inherit_config: bool = True, **kwargs):
        """Set configuration settings.

        Args:
            inherit_config (bool, optional): Set configurations for the given, and use existing configurations for the rest. Defaults to True.
        """
        if inherit_config:
            config = {**self.config, **kwargs}
        else:
            config = {**kwargs}

        self.__append(config)

So for now I feel it's better not change this pattern. But in the future (after we deprecate the dsp directory) we may want to have a centralized namespace like dspy.settings that is a collection of dynamic configs, e.g., dspy.settings.disable_litellm_logging() and dspy.settings.enable_verbose_logging().

@chenmoneygithub chenmoneygithub merged commit d69ec39 into stanfordnlp:main Nov 9, 2024
4 checks passed
@chenmoneygithub chenmoneygithub deleted the random-fix-4 branch December 27, 2024 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants