Config file | Distil Labs

The configuration file controls the training pipeline through four main sections: base, tuning, evaluation, and synthgen. Each section handles a specific aspect of the model training process.

File format

The configuration file supports two formats depending on how you interact with distil labs:

Webapp: Use JSON format (.json file)
API: Use YAML format (.yaml file)

Both formats are functionally equivalent—choose based on your workflow. Examples in this documentation show YAML, but the JSON equivalent is straightforward:

1 # YAML (API)
2 base:
3   task: information-extraction
4   student_model_name: Llama-3.2-3B-Instruct

1 // JSON (Webapp)
2 {
3   "base": {
4     "task": "information-extraction",
5     "student_model_name": "Llama-3.2-3B-Instruct"
6   }
7 }

Configuration structure

1 base:
2   # General parameters (task is required)
3   task: classification
4 
5 tuning:
6   # Fine-tuning parameters
7   num_train_epochs: 32
8 
9 evaluation:
10   # Evaluation parameters
11   num_few_shot_examples: 1
12 
13 synthgen:
14   # Synthetic data generation parameters
15   generation_target: 10000

Base configuration

General parameters relevant to the overall task.

Parameter	Type	Default	Description
`task`	`string`	required	Type of NLP task to be solved. See task types for available options.
`student_model_name`	`string`	`Llama-3.2-1B-Instruct`	Base model to use for the student model. This is the model we finetune for your use-case.
`teacher_model_name`	`string`	`Llama-3.3-70B-Instruct`	Teacher model used to generate synthetic data and from which we distil knowledge.
`random_seed`	`integer \| null`	`123`	Random seed used across distillib for reproducible random sampling.

Supported student models

Model	Value
Llama 3.2 1B Instruct	`Llama-3.2-1B-Instruct`
Llama 3.2 3B Instruct	`Llama-3.2-3B-Instruct`
Llama 3.1 8B Instruct	`Llama-3.1-8B-Instruct`
SmolLM2 135M	`SmolLM2-135M-Instruct`
Gemma 3 270M	`gemma-3-270m-it`
Gemma 3 1B	`gemma-3-1b-it`
Qwen3 0.6B	`Qwen3-0.6B`
Qwen3 1.7B	`Qwen3-1.7B`
Qwen3 4B	`Qwen3-4B-Instruct-2507`
Qwen3 8B	`Qwen3-8B`
IBM Granite 3.1 8B	`granite-3.1-8b-instruct`
IBM Granite 3.3 8B	`granite-3.3-8b-instruct`

Supported teacher models

Model	Value
DeepSeek R1	`deepseek.r1`
DeepSeek V3.1	`deepseek.v3.1`
Qwen3 235B A22B	`Qwen3-235B-A22B-Instruct-2507`
Qwen3 480B A35B Coder	`Qwen3-480B-A35B-Coder`
Qwen2.5 VL 72B	`Qwen2.5-VL-72B-Instruct`
Llama 3.1 405B Instruct	`Llama-3.1-405B-Instruct`
Llama 3.1 8B Instruct	`Llama-3.1-8B-Instruct`
Llama 3.3 70B Instruct	`Llama-3.3-70B-Instruct`
GPT OSS 20B	`openai.gpt-oss-20b`
GPT OSS 120B	`openai.gpt-oss-120b`
GPT OSS 120B Thinking	`openai.gpt-oss-120b-thinking`

Tuning configuration

Parameters controlling the finetuning of the student model.

Parameter	Type	Default	Description
`learning_rate`	`float`	`5e-5`	The initial learning rate for AdamW optimizer.
`learning_rate_scheduler`	`string`	`linear`	The scheduler type to use. Options: `cosine`, `linear`, `constant`.
`weight_decay`	`float`	`0.0`	Weight decay applied to all layers except bias and LayerNorm weights in AdamW optimizer.
`warmup_ratio`	`float`	`0.05`	Ratio of total training steps used for linear warmup from 0 to `learning_rate`.
`use_lora`	`boolean`	`true`	Whether to use LoRA for student training.
`lora_r`	`integer`	`64`	LoRA attention dimension (rank). Only used if `use_lora` is true.
`lora_alpha_multiplier`	`integer`	`1`	Alpha parameter for LoRA scaling is `lora_r * lora_alpha_multiplier`. Only used if `use_lora` is true.
`train_classification_as_textgen`	`boolean`	`false`	Train classification model as text-generation that generates class names. Only relevant for classification tasks.
`per_device_train_batch_size`	`integer`	`1`	Batch size per GPU/device for training.
`per_device_eval_batch_size`	`integer`	`1`	Batch size per GPU/device for evaluation.
`num_train_epochs`	`integer`	`4`	Total number of training epochs.
`train_eval_split`	`float`	`0.2`	Fraction of training data used for evaluation. Must be between 0 and 1 (exclusive).
`num_few_shot_examples_student`	`integer`	`0`	Number of few-shot examples when running student evaluation and tuning. If above 0, at least one example per class is used for classification tasks.

Evaluation configuration

Parameters used in teacher evaluation.

Parameter	Type	Default	Description
`num_few_shot_examples`	`integer`	`1`	Number of few-shot examples when running teacher evaluation. If above 0, at least one example per class is used for classification tasks.
`batch_size`	`integer`	`4`	(Deprecated) Batch size for model evaluation.

Synthetic generation configuration

Parameters for fine-grained control over synthetic data generation.

Parameter	Type	Default	Description
`generation_target`	`integer`	`10000`	Target number of synthetic examples to generate. For Closed-Book QA, this is calculated as `len(unstructured_data) * generation_per_unstructured_context`.
`generation_in_single_call`	`integer`	`4`	Number of examples to generate per teacher/LLM invocation.
`generation_iteration_size`	`integer`	`128`	Batch size for the generate-validate cycle.
`generation_per_unstructured_context`	`integer \| null`	`null`	Examples to generate per unstructured context. Only used with `question-answering-closed-book` task. Overwrites `generation_target` when set.
`num_positive_exemplars_per_generation`	`integer`	`2`	Number of in-context examples for the class/task being generated.
`num_negative_exemplars_per_generation`	`integer`	`2`	Number of in-context examples for classes not being generated. Only used for classification tasks.
`num_unlabelled_exemplars_per_generation`	`integer`	`2`	Number of unlabelled examples provided during each teacher invocation.
`validation_max_total_length`	`integer`	`10000`	Maximum total length (input + output) of generated examples in characters.
`validation_similarity_threshold`	`float`	`0.95`	Similarity threshold for deduplication. Generated data with similarity above this threshold to seed data are removed.
`validation_max_answer_length`	`integer`	`8192`	(Deprecated) Use `validation_max_total_length` instead.
`teacher_temperature`	`float`	`0.7`	Temperature for teacher output. Controls balance between predictability and creativity. Must be between 0.0 and 1.0.
`teacher_max_tokens`	`integer \| null`	`null`	Maximum tokens in the generated response.
`match_generated_distribution_to_seed`	`boolean`	`false`	Match generated data class distribution to seed data. Only used for classification tasks.
`num_distractor_context_blocks`	`integer`	`0`	Number of distractor context blocks per example. Setting above zero enables RAFT training.
`output_is_json`	`boolean`	`false`	Only generate synthetic data with valid JSON outputs. Only relevant for QA tasks.

Example configuration

Minimal configuration

1 base:
2   task: information-extraction
3   student_model_name: Llama-3.2-3B-Instruct
4   teacher_model_name: openai.gpt-oss-120b

Full configuration example

1 base:
2   task: question-answering-open-book
3   student_model_name: Qwen3-1.7B
4   teacher_model_name: openai.gpt-oss-120b
5   random_seed: 42
6 
7 tuning:
8   learning_rate: 1e-4
9   learning_rate_scheduler: cosine
10   use_lora: true
11   lora_r: 32
12   num_train_epochs: 3
13   train_eval_split: 0.15
14 
15 evaluation:
16   num_few_shot_examples: 2
17 
18 synthgen:
19   generation_target: 5000
20   generation_in_single_call: 8
21   teacher_temperature: 0.6
22   validation_similarity_threshold: 0.9

Model-specific notes

DeepSeek R1

When using deepseek.r1 as the teacher model, the recommended temperature range is 0.5 to 0.7. Configurations with temperatures outside this range will raise a validation error.

GPT OSS 120B Thinking

The openai.gpt-oss-120b-thinking model uses a medium reasoning effort setting by default for enhanced chain-of-thought capabilities.