Describe your approach to fine-tuning LLMs using enterprise datasets. What techniques do you use (SFT, LoRA, RLHF, DPO)?

Solution Overview

Applied a combination of Supervised Fine Tuning (SFT), LoRA, and RLHF to inject company-specific knowledge.

Steps

1. Dataset Preparation

   • Cleaned and anonymized documents

   • Removed PII

   • Structured Q/A pairs for SFT

   • Logged user queries for RLHF
     

2. Fine-tuning Methodology

   • Applied LoRA adapters to reduce training cost

   • Used lightweight DPO to align model tone

   • Reinforced reasoning quality using feedback loops
     

3. Evaluation

   • Benchmarked accuracy using golden datasets

   • Added regression tests

Results

• 40% improvement in domain accuracy

• Reduced hallucinations by 30%

• Lower latency with LoRA adapters