A production trace normally becomes useful when something breaks. An agent chooses the wrong tool, loops for too long, ignores a result, and an engineer opens LangSmith to reconstruct what happened. LangChain is now treating the same records as potential training material.

LangSmith Fine-Tuning, released in public beta, is built around smithtune. The CLI consumes LangSmith trajectories, turns selected runs into supervised fine-tuning data, sends training jobs to Fireworks or Baseten and brings the resulting evaluation back into LangSmith.

The useful unit is the trajectory

A trajectory contains the ordered messages, tool calls and tool outputs that made up an agent session. LangChain's format also tries to preserve the context available at each decision point. That matters for long-running agents, where the available tools or accumulated history may change while the task is in progress.

Exporting only the final conversation would lose part of that information. smithtune instead uses successful trajectories as examples of the behavior a student model should imitate.

Dataset preparation is handled inside the workflow. The CLI can pull traces from a LangSmith project, filter candidates, help create a rubric for deciding which runs are good enough for SFT and save the approved examples as a persistent dataset. It also checks sequence-length compatibility and creates training, validation and test splits.

Training is managed, but data quality is still your problem

smithtune plan exposes the model choice, number of examples and training settings before a run starts. smithtune train then submits supervised fine-tuning to Fireworks Managed SFT or Baseten Loops, with LoRA support described by LangChain for the prepared trajectories.

That removes GPU provisioning from this particular workflow, not the cost or responsibility of training. With Baseten, for example, jobs run against the customer's own workspace and API key, and Baseten says training and inference are billed to that account. Its Loops product is also still in early access.

More importantly, LangChain explicitly recommends trying harness improvements before reaching for fine-tuning. SFT is presented as a better fit when an agent performs repeated tasks, keeps making recurring mistakes and already has examples demonstrating the desired behavior.

smithtune evaluates the checkpoint before deployment

The evaluation step uses held-out trajectories to compare the tuned checkpoint with its base model. smithtune replays actions from those examples, scores the predictions against recorded reference behavior and sends the comparison into LangSmith for inspection.

If the new model is acceptable, smithtune deploy can then serve it through the selected provider. If it is not, the intended loop is to revise the dataset or training configuration and run the comparison again.

LangChain's own numbers come with an important warning

LangChain says it used the workflow on two internal agent workloads. On a subset of its IssueBench benchmark, a fine-tuned Kimi K3 scored 96 versus 90 for the base Kimi K3 and 87 for GPT-5.6 Sol. In an internal OpenSWE Review evaluation, fine-tuning Qwen-3.8-27B increased F1 from 48.9% to 53.7% while reducing model calls by 29.8% and tool requests by 29.4%.

Those are company-run evaluations rather than independent benchmarks. They are also accompanied by a more useful failure case: LangChain says an earlier, less carefully selected training set actually reduced F1 after SFT. The team changed its curation process and reviewed the traces more selectively before the later run improved.

That is a fairly good summary of what smithtune can and cannot automate. The CLI can connect collection, training, evaluation and deployment. It cannot turn bad examples into a good specialist model merely because the pipeline is convenient.

LangSmith is closing its agent-improvement loop

LangSmith already occupied the observation and evaluation side of agent engineering. Fine-Tuning extends that workflow into post-training: production behavior becomes a source of examples, selected examples become a training dataset, and the new checkpoint is tested against held-out behavior before it is served.

The public beta requires a LangSmith account containing agent traces, the smithtune CLI and credentials for either Fireworks or Baseten. LangChain released it on September 24.