UI Overview

This page describes the user interface of ChainForge, including all buttons in the toolbar. Below is a screenshot of the interface. The interface is loaded with an experiment (a 'flow') testing a chat LLM's tendency to change to an incorrect answer if the user tells it that it's wrong:

There's a toolbar on the top, and a number of blocks —what we call 'nodes' —connected by lines. Outputs of nodes are on the right; inputs are on the left. Data generally flows left-to-right.

In the experiment, two Items Nodes (a.k.a. Data Loaders) provide input to a Prompt Node and a Chat Turn Node, respectively. The output of the Prompt Node feeds into the Chat Turn Node to continue a conversation with 3 queried LLMs. The responses from the Chat Turn node are then fed into an LLM Scorer that returns boolean values true or false. The scores are finally fed into a Vis Node to plot results. Here, we see that ChatGPT (GPT3.5) is the most robust against 'changing its mind' when told that it's wrong, for the tried inputs.

Below we describe the toolbar and node UI. If you'd like to know more about specific capabilities, visit Nodes or see the pages under Features in the left side-bar.

Toolbar

Button	Function	Onclick event
	View all nodes currently available. Select a node and it will appear in the interface. For specific details on each node, visit the Nodes page.
	Share your evaluation flows with others. You can export your flow as a cforge file (JSON) by clicking the Export button. All cache'd LLM responses and evaluation results are exported with your current flow.
	Upload a ChainForge file to view and/or modify a previous flow.
	The web version of ChainForge includes a Share button. Simply click Share to generate a unique link for your flow and copy it to your clipboard! Note: To prevent abuse, you can only share up to 10 flows at a time, and each flow must be <5MB after compression. If you share more than 10 flows, the oldest link will break, so make sure to always Export important flows to cforge files, and use Share to only pass data ephemerally.
	Generate a fresh programming environment. Once you click 'Confirm,' your previous nodes will be erased and you'll have a blank TextFields Node and sample Prompt Node.
	View example flows to give you a sense of what's possible with ChainForge. Click 'Try me' to load an example. Click the tab OpenAI Evals to see 188 example flows generated from the OpenAI evals benchmarking suite.
	Set your API keys! If you prefer to not worry about this everytime you open ChainForge, we recommend that save your OpenAI, Anthropic, and/or Google PaLM API keys to your local environment. For more details, see installation page.

Connecting and disconnecting nodes

Connect outputs to inputs by clicking the handle bead and dragging to the input bead:

You can delete edges by hovering over the edge, clicking it, and hitting the Delete key:

Finally, you can rename nodes by clicking on their labels; and you can delete nodes by clicking the X in the top-right corner. For info on what nodes are available, see the Nodes page.

Info

We realize it's hard to select edges. This is a difficulty with ReactFlow, the library we're using, that we hope to resolve soon.