AI for ChainForge

Evaluating LLM behavior is tough, often involving tedious creation of prompts or input data. What if you could generate input data—from text fields to entire tables—at will? What if you could smart-extend data? ChainForge has Generative AI features that allow you to do just that.

Note

Generative AI features currently require an OpenAI API key to use. It queries GPT-4o and GPT-4o-mini models.

✨ Generate or Extend Tabular Data 🗂

Synthesize tabular data using the new Replace and Extend buttons in the Tabular Data node. Click the purple sparkly button ✨:

This will generate (setting the rows=10):

Note

AI can hallucinate, so be sure to check the results.

Extend existing tables, both rows and columns

You can also extend on rows and columns! The latter is interesting: you could import your own data, and then use the AI to generate new rows or columns.

Extending on columns is robust to the length of the table (but it may take awhile if you have a large table, so get that popcorn ready! 🍿).

✨ Generate or Extend Input Data

TextFields and Items Nodes also include a purple sparkly button ✨ when AI Support is enabled. Generate new data, or extend existing data, by meta-prompting or just clicking Extend:

Generate prompt templates

You can use this to generate or extend prompt templates, too. It will try to be consistent with variable names:

Currently, you must use the term prompt in your meta-prompt in order to generate prompts. Generating prompts is an experimental feature and may not always work. However, just click to regenerate and try again!

More soon!

We are working on more generative AI features, including ChainBuddy and EvalGen.