Best Paper Award 2024

During the 6th Annual Meeting 2024, the paper "Large Language Models to Generate System-Level Test Programs Targeting Non-functional Properties" by Denis Schwachhofer, Peter Domanski, Steffen Becker, Stefan Wagner, Matthias Sauer, Dirk Pflüger, and Ilia Polian of project P3 and P5 received the GS-IMTR Best Paper Award 2024. The paper was published at 37th IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems.

Congratulations!

Abstract

System-Level Test (SLT) is essential for testing integrated circuits, focusing on functional and non-functional properties of the Device under Test (DUT). Traditionally, test engineers manually create tests with commercial software to simulate the DUT’s end-user environment. This process is both time-consuming and offers limited control over non-functional properties. This paper proposes Large Language Models (LLMs) enhanced by Structural Chain of Thought (SCoT) prompting, a temperature schedule, and a pool of previously generated snippets to generate high-quality code snippets for SLT. We repeatedly query the LLM for a better snippet using previously generated snippets as examples, thus creating an iterative optimization loop. This approach can automatically generate snippets for SLT that target specific non-functional properties, reducing time and effort. Our findings show that this approach improves the quality of the generated snippets compared to unstructured prompts containing only a task description.