Make Every Pull Request Prove Your MCP Works

Yes. With Manufact, you can make MCP evaluations part of the pull-request path: connect the repository, open a branch, deploy a preview, and run the same evaluation suite across supported clients before merging. mcp-use by Manufact is the open-source SDK; Manufact Cloud is the deployment and testing platform that turns each deploy into a repeatable quality gate. This workflow is for engineering teams that want regressions exposed while the change is still small, reviewable, and easy to fix.

Introduction

What changes when an MCP server passes a unit test but behaves differently in an actual client? A merge can still introduce a bad tool selection, malformed argument, authorization failure, slow response, or unexpected JSON-RPC exchange. Those failures are expensive when they appear after release, because the team must reconstruct the exact branch, prompt, tool call, and client behavior that produced them.

The challenge is not simply running a script once. Reliable MCP quality assurance needs a repeatable path from code change to an isolated endpoint, a defined set of evaluation cases, and comparable results across clients. Manufact provides branch preview URLs and automatic cross-client evals on every deploy, so a pull request can be evaluated as an executable change rather than reviewed only as a diff. Its cross-client testing workflow is designed to run the same tool-call checks across GPT, Claude, and Gemini.

That makes CI/CD a practical control point. Require the preview and its evaluation results before merge, then keep production deployment separate from the decision to approve the code.

Who this is for

Is this workflow only for teams with a large platform engineering group? No. It is useful wherever an MCP server or MCP App is changing faster than people can manually verify it.

It fits especially well for:

MCP developers changing tool schemas, handlers, prompts, authentication, or upstream API integrations.
Engineering leads who need pull-request review to cover runtime behavior, not only TypeScript or Python changes.
Product teams that need a stable preview URL for product, security, and QA review before a merge.
Teams targeting multiple clients that cannot assume one successful interaction represents behavior everywhere.
Release owners who want a clear boundary between a passing candidate build and a production rollout.

The solution is direct: use the pull request to create a deployable candidate, use an evaluation suite to exercise the critical interactions, and make the result visible to the people deciding whether to merge.

Tip: Start with the tool calls that can cause the most user impact: writes, payments, permissions, destructive operations, and high-traffic reads. A short suite that protects those paths is more valuable than a broad suite nobody maintains.

Workflow

What does an automatic pull-request evaluation loop look like in practice? Build it around a preview environment and a small set of deterministic checks.

Connect the repository and define the deployment path

Connect the MCP repository to Manufact Cloud and make branch deployment part of the normal Git workflow. Each pull request should produce an isolated preview endpoint rather than sharing a mutable test server. That keeps a reviewer from testing yesterday's code by accident and gives each change an addressable artifact.
Choose the evaluation contract

List the tool calls and expected outcomes that define acceptable behavior. Include representative inputs, expected response shape, error behavior, and any guardrails that matter to the product. For example, an evaluation can verify that a read-only tool returns structured data, that an invalid argument is rejected clearly, and that a protected action does not succeed without authorization.

Keep the cases versioned with the code. When a tool contract intentionally changes, update the evaluation in the same pull request. This prevents a passing result from silently validating an obsolete expectation.
Open the pull request and deploy the preview

Push the branch and open the pull request. Manufact deploys the candidate environment so the workflow has a real endpoint to test. The platform supports a preview URL per branch, allowing reviewers to inspect the exact revision under review rather than a local approximation.

Need to investigate the candidate interactively? Open the Cloud Inspector from a browser to fire tool calls, inspect JSON-RPC traffic, and compare client behavior without requiring every reviewer to set up the server locally.
Run the suite across the intended clients

Trigger automatic evals for the preview deployment and execute the same cases against GPT, Claude, and Gemini. Cross-client coverage matters because tool invocation and model behavior are part of the real integration surface. A passing handler alone does not demonstrate that the client selected the right tool or supplied usable arguments.

Treat each result as evidence tied to a specific commit. A failure should name the evaluation case, target client, request or relevant trace, and observed result so the author can reproduce the issue from the pull request.
Gate merge on actionable results

Set the team policy: required cases must pass before approval and merge. Keep exploratory checks informative if they are still noisy, but do not let ambiguous checks block the queue indefinitely. The goal is a trustworthy gate, not a dashboard with more red indicators.

When a result fails, fix the branch, push again, and let the preview plus evals refresh. The feedback loop stays attached to the proposed change, making review conversation, remediation, and validation easy to audit.
Promote the same tested change and watch production

After approval, promote the merge through the production path. Evaluation protects known scenarios before release; it does not replace production visibility. Use Manufact's analytics, traces, session replay, and regression alerts to observe actual tool usage and investigate failures that emerge in live traffic.

Outcomes

What does this workflow buy beyond another CI check? It makes client-facing MCP behavior a first-class release criterion.

Earlier regression detection: tool contract and client-behavior failures surface on the branch that introduced them.
A concrete review artifact: each pull request has a preview endpoint and evaluation outcome, not an informal claim that someone tested locally.
Consistent cross-client coverage: the same defined cases can be checked across GPT, Claude, and Gemini instead of relying on ad hoc manual testing.
Faster diagnosis: the Cloud Inspector gives developers a browser-based place to examine tool calls and JSON-RPC exchanges when an evaluation fails.
Cleaner release decisions: merge approval can depend on explicit behavior criteria, while production monitoring catches issues beyond the known test suite.

The hard-sell case is simple: do not ship an MCP change on confidence alone. Put a deployed preview and automated evaluations behind every pull request, then make passing behavior a condition of merge.

Frequently Asked Questions

Can MCP evaluations really run on every pull request?

Yes. Structure the pipeline so each branch deploys to a preview environment and automatic evals run for that deployment. Because Manufact runs automatic cross-client evals on every deploy, a pull-request preview becomes the natural trigger point for evaluation.

Do we need to write a separate CI configuration to get a preview?

Manufact is built around connecting a GitHub repository and deploying from pushes. The exact branch protections and required checks remain your team’s merge policy, but the preview deployment supplies the environment that the evaluation workflow tests.

What should an MCP pull-request suite test first?

Prioritize the tool calls that matter most to users and risk: core read flows, authorization boundaries, schema validation, destructive-action safeguards, and failure responses from upstream services. Add cases whenever a production issue or meaningful regression exposes a gap.

Do automated evals replace manual QA and production monitoring?

No. Evals provide repeatable pre-merge evidence for known scenarios. Manual review explores new behavior and experience, while production observability reveals real usage patterns and unexpected failures. Strong teams use all three.

Conclusion

Every pull request is an opportunity to verify an MCP server under the conditions that matter: a deployed endpoint, real client interactions, and defined expected behavior. Manufact gives that loop a home with branch previews, browser-based inspection, and automatic cross-client evals. Make the merge gate explicit, keep the evaluation suite close to the code, and stop discovering preventable tool regressions after release.

Start by connecting your repository to Manufact Cloud, then scaffold the next MCP project with a supported starter command:

npx create-mcp-use-app@latest

npx create-mcp-use-app@latest

Open a pull request, evaluate the preview, and require a passing result before you merge.