ArchLite: Do Static Microarchitecture Decisions Really Need Large Models?

In recent years, a hot direction in systems research has been Systems for AI: how to make large-model training and inference faster, more memory-efficient, and easier to deploy. ArchLite focuses on the opposite question: Can AI help systems themselves make decisions? Around April this year, I read the ASPLOS 2026 best paper PF-LLM: Large Language Model Hinted Hardware Prefetching, and a question came up: why PF-LLM, but not PF-DNN? PF-LLM indeed achieved good results on prefetching decisions, but its model size and training cost are also nontrivial. This led to the machine-learning course project ArchLite, a comparative study of model scale for static load-level prefetching.

More specifically, ArchLite tries to answer one question:

For static, local, structured microarchitecture decisions, do we really need large language models?

This question was inspired by PF-LLM. PF-LLM models hardware prefetching decisions as prediction tasks over code context. ArchLite does not try to strictly reproduce PF-LLM. Instead, it uses PF-LLM as a case study and investigates model scale itself: on the same locally generated data and labels, it compares rule-based or linear models, compact DNNs, and Qwen2.5 LoRA.

Task: Predicting Prefetch Hints from Assembly Context

The concrete task chosen by ArchLite is static load-level prefetching. Each sample corresponds to a static load PC. The input is the assembly context around that load instruction, and the output is a structured label with three fields:

1
2
3
4
5
{
  "PF Sel": "...",
  "PF Degree": "...",
  "Filter": "..."
}

Where:

  • PF Sel: which type of prefetcher to choose, such as stream, sms, ip_stride, or sandbox;
  • PF Degree: the prefetch degree;
  • Filter: whether to use a filtering strategy.

This task is different from general code understanding. It does not require code generation or cross-file reasoning. It is a local supervised-learning problem with a limited output space. Therefore it is well suited for testing whether the scale advantage of large models is truly necessary.

How the Data Is Generated

ArchLite’s labels are not manually annotated. They are generated through system simulation.

The overall process is:

1
2
3
4
5
6
7
8
9
10
11
GAPBS graph workload
        ↓
Execution trace
        ↓
Extract static load instructions and assembly context
        ↓
ChampSim grid search over prefetch configurations
        ↓
Use the configuration with the lowest AMAT for each load PC as the label
        ↓
Train and compare LR, DNN, and Qwen2.5 LoRA

GAPBS is used because graph workloads often have irregular memory access patterns, making prefetching decisions nontrivial. ChampSim provides a controllable microarchitecture simulation environment. For each static load PC, ArchLite compares AMAT under different prefetcher configurations and selects the configuration with the lowest AMAT as the supervised label.

The project constructs three datasets, differing in sample-filtering rules:

Dataset Rule Balanced train Balanced test Original test
margin002 margin >= 0.02, min count >= 3 556 88 192
nomargin no margin, min count >= 3 1240 232 508
mincount1_nomargin no margin, min count >= 1 4112 592 2355

The balanced test split avoids majority-class dominance hiding model weaknesses, while the original test split preserves the natural distribution.

Model Comparison

ArchLite compares four types of models:

  1. Majority baseline: always predicts the most common label in the training set;
  2. Logistic Regression: hashed TF-IDF features over assembly tokens;
  3. Compact DNN: a small three-head MLP predicting the three fields;
  4. Qwen2.5 LoRA: LoRA fine-tuning of Qwen2.5-Coder-0.5B-Instruct.

The most important results come from the largest dataset, mincount1_nomargin:

Split Model PF Sel PF Degree Filter Joint
balanced Logistic regression 0.5389 0.6402 0.6199 0.2584
balanced DNN 0.6622 0.6993 0.6554 0.4206
balanced Qwen2.5 LoRA 0.6858 0.6520 0.6436 0.4291
original Logistic regression 0.5495 0.6611 0.6102 0.2887
original DNN 0.6972 0.7231 0.6688 0.4599
original Qwen2.5 LoRA 0.7227 0.6985 0.6450 0.4603

Qwen2.5 is clearly stronger than Logistic Regression, but it does not significantly dominate the compact DNN. On the balanced split, Qwen’s joint accuracy is only 0.0085 higher than DNN. On the original split, the two are nearly tied.

This is ArchLite’s core observation: for such local, structured system tasks with clear supervised signals, large models are not necessarily the most cost-effective choice.

A More Interesting Finding: Local Context Is Better

ArchLite also performs a context-window ablation. By default, the model can see the full assembly context. But if we keep only the local window around the target load, the result actually improves.

On the largest balanced test split:

Context window PF Sel PF Degree Filter Joint
full 0.6655 0.6943 0.6672 0.4257
8 lines each side 0.7044 0.7078 0.6875 0.4764

In other words, by looking only at eight assembly lines before and after the target load, the compact DNN reaches 0.4764 joint accuracy, exceeding full-context DNN and also exceeding the current Qwen2.5 LoRA result of 0.4291.

This suggests that effective signals for this task may be concentrated near the target instruction. For microarchitecture decisions, more context is not always better. A suitable local representation may match the problem structure more closely.

AMAT Proxy: Not Just Classification Accuracy

Classification accuracy only tells us whether the model predicts the same label. But system tasks ultimately care about performance. Therefore ArchLite also computes an AMAT-level proxy: mapping the predicted prefetch configuration back to ChampSim results and estimating how much oracle prefetching benefit it recovers.

Results:

Model Predicted AMAT Recovery vs no-prefetch
Majority 173.1443 0.2252
Logistic regression 147.6176 0.4916
DNN 127.4078 0.6966
Qwen2.5 LoRA 126.8828 0.6960

DNN and Qwen2.5 have almost identical AMAT recovery. This further shows that the compact DNN is not merely close in classification metrics; it also achieves similar downstream performance proxy metrics.

ArchLite’s Conclusion

ArchLite is not saying that LLMs have no value for architecture tasks. Large models may be more suitable for cross-function reasoning, source-code semantic understanding, low-sample transfer, explanation generation, and similar tasks.

It emphasizes a finer judgment:

When the task is local, the output space is limited, and labels can be obtained through simulation or measurement, model scale should be treated as a system design parameter, not as something that is bigger by default.

For tasks like static load-level prefetching, compact DNNs can already capture many learnable signals while being cheaper to train, infer, and deploy.

Limitations and Future Work

Current ArchLite is still a local-data study rather than a complete industrial system. Its main limitations include:

  • workloads mainly come from GAPBS and do not yet cover SPEC, server workloads, or broader programs;
  • labels come from ChampSim simulation and may differ from real hardware behavior;
  • the current split is closer to held-out input generalization than strict cross-program generalization;
  • Qwen, DNN, and LR have different hyperparameter search spaces, so the results should be interpreted as a controlled practical comparison rather than an exhaustive search.

Future work can expand to more workloads, stricter cross-program tests, more realistic hardware validation, and more microarchitecture decisions, such as cache bypass, branch behavior, memory dependence, and load/store scheduling.

Summary

ArchLite’s core value is not proposing a new prefetcher. It raises a system design question: does AI for Systems always need large models?

At least on this static prefetching decision task, the answer is not absolute. Qwen2.5 LoRA is effective, but a compact DNN is already very close and even stronger under a local-context setting.

This gives AI for Systems a practical lesson: before introducing LLMs into system optimization, we should carefully examine task granularity, label source, output space, and deployment cost. Many times, the truly suitable model is not the largest one, but the one that best matches the structure of the problem.