Overview

Whether you are looking to classify text, answer questions, interact with internal tools, or solve other language tasks, our step-by-step workflow will take you from initial concept to production-ready model. Let’s dive in!

Authentication

First, authenticate with the distil labs platform:

1
distil login

Step 1: Create a model

Register a new model to track your experiment:

1
# Returns your model ID
2
distil model create my-model-name

You can list all your models with:

1
distil model list

Step 2: Task selection and data preparation

Begin by identifying the specific task you want your model to perform. Different tasks require different approaches to data preparation and model configuration.

Learn more about task selection →

Prepare your data and configuration files according to your chosen task type. A training job requires the following files in a directory:

Learn more about data preparation →

Upload your data to the model (from your local ./data directory):

1
distil model upload-data <model-id> --data ./data

Step 3: Teacher evaluation

Before training your specialized small model, validate whether a large language model can accurately solve your task with the provided examples. If the teacher model can solve the task, the student model will be able to learn from it effectively. Learn about teacher evaluation →

1
# Start teacher evaluation
2
distil model run-teacher-evaluation <model-id>
3
4
# Check status
5
distil model teacher-evaluation <model-id>

Step 4: Model training

Once your teacher evaluation shows satisfactory results, train your specialized small language model using knowledge distillation.

Understand the model training process →

1
# Start training
2
distil model run-training <model-id>
3
4
# Check status
5
distil model training <model-id>

Step 5: Download your model

Once training is complete, download your model:

1
distil model download <model-id>

Step 6: Model deployment

Deploy your trained model locally or using distil labs inference for immediate integration with your applications.

Explore deployment options →

If you decide to deploy locally, download the model and setup inference with Ollama:

1
ollama create my-model -f Modelfile
2
ollama run my-model

Query the model with the OpenAI-compatible API:

1
from openai import OpenAI
2
3
client = OpenAI(
4
    base_url='http://localhost:11434/v1',
5
    api_key='ollama',
6
)
7
8
response = client.chat.completions.create(
9
    model="my-model",
10
    messages=[
11
        {"role": "system", "content": "You are a helpful assistant."},
12
        {"role": "user", "content": "Hey, can you help me?"},
13
    ],
14
)
15
print(response.choices[0].message.content)

Next steps

You’ve successfully trained and deployed a specialized small language model! For more details, explore:

• Tutorials for complete end-to-end examples
• Deployment options for production deployment