Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: twitter actions not triggering #903

Merged
merged 1 commit into from
Dec 7, 2024
Merged

fix: twitter actions not triggering #903

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion packages/client-twitter/src/interactions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,10 @@ Thread of Tweets You Are Replying To:

{{actions}}

# Task: Generate a post in the voice, style and perspective of {{agentName}} (@{{twitterUserName}}). Include an action, if appropriate. {{actionNames}}:
# Task: Generate a post in the voice, style and perspective of {{agentName}} (@{{twitterUserName}}). You MUST include an action if the current post text includes a prompt that is similar to one of the available actions mentioned here:
{{actionNames}}

Here is the current post text again. Remember to include an action if the current post text includes a prompt that asks for one of the available actions mentioned above (does not need to be exact):
{{currentPost}}
` + messageCompletionFooter;

Expand Down
57 changes: 36 additions & 21 deletions packages/core/src/generation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -778,6 +778,7 @@ export async function generateMessageResponse({
context,
modelClass,
});

// try parsing the response as JSON, if null then try again
const parsedContent = parseJSONObjectFromText(response) as Content;
if (!parsedContent) {
Expand Down Expand Up @@ -884,33 +885,41 @@ export const generateImage = async (
});

// Add type assertion to handle the response properly
const togetherResponse = response as unknown as TogetherAIImageResponse;
const togetherResponse =
response as unknown as TogetherAIImageResponse;

if (!togetherResponse.data || !Array.isArray(togetherResponse.data)) {
if (
!togetherResponse.data ||
!Array.isArray(togetherResponse.data)
) {
throw new Error("Invalid response format from Together AI");
}

// Rest of the code remains the same...
const base64s = await Promise.all(togetherResponse.data.map(async (image) => {
if (!image.url) {
elizaLogger.error("Missing URL in image data:", image);
throw new Error("Missing URL in Together AI response");
}
const base64s = await Promise.all(
togetherResponse.data.map(async (image) => {
if (!image.url) {
elizaLogger.error("Missing URL in image data:", image);
throw new Error("Missing URL in Together AI response");
}

// Fetch the image from the URL
const imageResponse = await fetch(image.url);
if (!imageResponse.ok) {
throw new Error(`Failed to fetch image: ${imageResponse.statusText}`);
}
// Fetch the image from the URL
const imageResponse = await fetch(image.url);
if (!imageResponse.ok) {
throw new Error(
`Failed to fetch image: ${imageResponse.statusText}`
);
}

// Convert to blob and then to base64
const blob = await imageResponse.blob();
const arrayBuffer = await blob.arrayBuffer();
const base64 = Buffer.from(arrayBuffer).toString("base64");

// Convert to blob and then to base64
const blob = await imageResponse.blob();
const arrayBuffer = await blob.arrayBuffer();
const base64 = Buffer.from(arrayBuffer).toString('base64');

// Return with proper MIME type
return `data:image/jpeg;base64,${base64}`;
}));
// Return with proper MIME type
return `data:image/jpeg;base64,${base64}`;
})
);

if (base64s.length === 0) {
throw new Error("No images generated by Together AI");
Expand Down Expand Up @@ -976,7 +985,13 @@ export const generateImage = async (
) {
targetSize = "1024x1024";
}
const openai = new OpenAI({ apiKey: apiKey as string });
const openaiApiKey = runtime.getSetting("OPENAI_API_KEY") as string;
if (!openaiApiKey) {
throw new Error("OPENAI_API_KEY is not set");
}
const openai = new OpenAI({
apiKey: openaiApiKey as string,
});
const response = await openai.images.generate({
model,
prompt: data.prompt,
Expand Down
Loading