Model Integration Contracts

What are MICs?

MIC packages a model’s real usage code, a smaller version of the model for fast runs, and reviewed expected results. Transformers CI runs that usage and reports when behavior changes.

You or your agent can drive the whole workflow: create a MIC, run checks, investigate failures, and propose fixes.

01 / Understand

How do we create the shrunk model?

An agent can drive this entire workflow with tmic. Agent mode returns structured questions for review and upload decisions.

tmic run HuggingFaceTB/SmolLM-135M --mode agent --regenerate --namespace huggingface-mic

The agent resumes the same command with answers as needed. Explore each step below.

01 / Understand

Inside usage.md

The code in usage.md is a fully valid Transformers Python script: generate text, then take one training step using the same model.

## Generate
```py runnable:generate
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "HuggingFaceTB/SmolLM-135M"
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"
model = AutoModelForCausalLM.from_pretrained(model_id)
prompts = ["The capital of France is",
           "The capital of Germany is"]
inputs = tokenizer(prompts, return_tensors="pt", padding=True)
outputs = model.generate(**inputs, max_new_tokens=4,
    do_sample=False, pad_token_id=tokenizer.eos_token_id)
new_tokens = outputs[:, inputs["input_ids"].shape[1]:]
print(tokenizer.batch_decode(new_tokens, skip_special_tokens=True))
```

## One training step
```py runnable:generate:2
answers = [" Paris", " Berlin"]
tokenizer.padding_side = "right"
batch = tokenizer(
    [p + a for p, a in zip(prompts, answers)],
    return_tensors="pt", padding=True)
labels = batch["input_ids"].masked_fill(
    batch["attention_mask"] == 0, -100)

model.train()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
loss = model(**batch, labels=labels).loss
loss.backward()
optimizer.step()
print(f"loss: {loss.item():.4f}")
```

01 / Understand

Inside assertions.json

For the prompt “The capital of France is”, this slim model’s recorded output starts with “ Paris”.

{
  "outputs": {
    "generated_tokens": [[7042, 7042, 7042, 7042]],
    "generated_text": [" Paris Paris Paris Paris"],
    "next_token_logits": [[12.8125, 3.046875, 7.59375, 5.03125]],
    "training": {
      "loss": 0.2566872537136078,
      "updated": true
    }
  }
}

02 / Use in CI

How MICs are used

A very fast CPU job (few seconds per model) runs the small models on PRs and on main. No GPU is needed for these checks.

The job is currently observational: failures are reported without blocking the PR.

03 / Investigate

A MIC failed in CI. What next?

Ask your agent to inspect the failure and reproduce it with the recorded framework revision, fleet snapshot, and runtime.

03 / Investigate

Give your agent the failing case.

Share the Run or Trace link. The agent can inspect the assertion, fetch pinned references, reproduce the failure, and propose a fix with evidence.

Output: logits · comparison against the reviewed baseline
Record: contract + target + variant + platform + framework SHA + fleet SHA
Open: Trace · Run · Baseline · Contract

04 / Agents on the Hub

Improve discoverability for agents through the HUB

A proposed link from the original Hub model to its MIC would give an agent a tested starting point, instead of another snippet to debug.

Explore MIC

Try it. Explore the models. Follow the results.

Give these links to your agent to get started.