In One Sentence

LoRA is a technique that specializes a large AI model for your business by attaching small "add-on learning modules" to the parts that matter — without rebuilding the entire model.

Think of it as renovating a house instead of tearing it down and rebuilding. Compared to a full rebuild (full fine-tuning), it dramatically reduces both cost and turnaround time.

Why It Matters

General-purpose LLMs such as ChatGPT are remarkably capable, but out of the box they cannot speak "your company's language."

  • They misuse industry-specific terminology and abbreviations
  • Their output does not follow your response formats or writing style
  • They cannot reflect the decision patterns of your specific operations

One way to solve this is fine-tuning (additional training). However, LLMs have billions to hundreds of billions of parameters, and retraining an entire model requires monopolizing large numbers of high-end GPUs for long periods. For most companies, that cost is simply not realistic.

LoRA (Low-Rank Adaptation) was created to solve exactly this cost problem.

How It Works

Getting slightly technical, the idea behind LoRA is simple.

Fine-tuning is fundamentally the act of updating the huge weight matrices inside a model. LoRA freezes those huge matrices entirely and never touches them — instead, it expresses only the delta (the change) using separate, much smaller matrices.

The key is approximating that delta as the product of two thin, low-rank matrices (A and B). For example, directly updating a 4096×4096 weight matrix means adjusting roughly 16.78 million parameters. With rank-8 LoRA, you only need two 4096×8 matrices — about 65,000 parameters. That is less than 1% of the original.

This structure yields several properties that are genuinely useful in practice.

  • Training costs drop sharply: far fewer parameters to update means far less GPU memory and training time
  • The base model stays intact: because it is frozen, general capabilities are unlikely to degrade
  • Adapters are swappable: the training result is a small file of a few dozen megabytes, so one base model can serve multiple roles — a "sales" adapter, a "support" adapter, and so on

More recently, QLoRA — which quantizes the base model to compress memory before LoRA training — has become widespread, making it possible to tune models with tens of billions of parameters on a single GPU.

When to Use It

LoRA is the right tool when you want to change the behavior of the model itself.

  • Standardize output style, format, and tone to your company's specifications
  • Enforce consistent use of industry-specific terminology and phrasing
  • Improve accuracy on specific tasks (classification, extraction, template-based document generation, etc.)
  • Manage multiple model variations for different business functions at low cost

When Not to Use It

There are also problems LoRA cannot solve. Getting this wrong is the classic pattern of spending money and seeing no results.

  • You want the model to answer with up-to-date information or internal documents: fine-tuning is poor at injecting knowledge and weak at keeping it current. That is the domain of RAG (retrieval-augmented generation)
  • You cannot prepare training data: even LoRA needs hundreds to thousands of high-quality supervised examples. Starting without data preparation leads to failure
  • Prompt engineering already solves it: try prompt design and few-shot examples first. Tuning should be the last resort

RAG vs. SFT vs. LoRA

Three commonly confused terms, sorted out.

Approach What it does Best suited for
RAG Retrieves external documents and passes them to the model Answering from internal knowledge and current information
SFT Trains model behavior on supervised data (the general approach) Response formats and task accuracy
LoRA A training technique that makes SFT affordable The same goals as SFT, with far less compute

In short: RAG and SFT differ in purpose, while SFT and LoRA share a purpose but differ in method. In practice, many teams divide the roles — "knowledge goes to RAG, behavior goes to LoRA-based SFT" — and use both together.

Use Cases for Small and Mid-Sized Companies

  • Customer support: build a model that drafts first-response replies in your company's tone, trained on your best past support logs
  • Manufacturing: teach the writing conventions of reports and specifications to standardize document drafting
  • Professional services: specialize the generation of routine documents (contract drafts, application forms) to your firm's established formats
  • E-commerce: generate product descriptions at scale, consistently aligned with your brand's tone and style

What these all have in common: they teach the model a fixed pattern of work, not open-ended knowledge.

Our Perspective

We do not recommend starting with LoRA fine-tuning right away.

For most companies, the realistic sequence is: (1) start with prompt design, (2) externalize internal knowledge with RAG, (3) build evaluation data so accuracy can be measured quantitatively, and (4) only then consider SFT/LoRA for the behavioral gaps that remain.

LoRA is a powerful technique, but it is only a means to an end. What matters most is determining upfront whether your problem is a knowledge problem or a behavior problem — and that assessment is exactly where we help our clients begin.