Model tuning allows you to enhance the quality of responses and tailor them to specific domains beyond just prompt design.

Prompt Design

Prompt design lets you tune a gen AI model using natural language without any ML background.

To enhance the model’s performance, you can provide context and examples to guide its responses.

Prompt design does not alter the parameters of the pre-trained model. Instead, it improves the model’s ability to respond appropriately by teaching it how to react.

One benefit of prompt design is that it enables rapid experimentation and customization. Another benefit is that it doesn’t require specialized machine learning knowledge or coding skills, making it accessible to a wider range of users.

However, this is a brittle approach, as changing a single word or model can change the output significantly.

Fine Tuning

One way to address prompt crafting problems is to fine tune the models using your own data.

However, fine-tuning the entire model can be impractical due to the high computational resources, cost, and time required.

LLMs have a vast number of parameters, making it computationally demanding to update every weight.

Parameter-efficient Tuning

This involves making smaller changes to the model, such as training a subset of parameters or adding additional layers and embeddings.

Examples:

• Adapter Tuning: supervised tuning that lets you use as few as one hundred examples to improve model performance
• Reinforcement tuning: unsupervised reinforcement learning with human feedback.
• Distillation: more technical tuning technique, enables training smaller task-specific models with less training data, and lower serving costs and latency than the original model. Exclsive to Google Cloud.

Distillation

This technology transfers knowledge from a larger model to a smaller model to optimize performance, latency, and cost.

The rationale is to use a larger teacher model that trains smaller student models to perform specific tasks better and with improved reasoning capabilities.

The training and distilling process uses labeled examples and rationales generated by the teacher model to fine-tune the student model.

Rationales are like asking the model to explain why examples are labeled the way they are.