Black-box GPU kernel performance optimization with AI agents
OpenBMB used an AI agent to autonomously perform black-box kernel optimization on FlashMLA on an NVIDIA B200 over approximately 22 hours. Without access to official source code, documentation, or profiling information, the agent achieved a 1.098x performance boost (472 µs vs. 518 µs, a 9.8% speedup) solely through execution and benchmarking.
Reddit r/LocalLLaMA/u/Equivalent-Grass-527
ForgeKernel harnessNVIDIA B200
Added 2026-11-03 · originally published 2026-11-02
Steps
- 1Set up the execution and benchmarking environment for the target black-box kernel (such as the official FlashMLA implementation).
- 2Deploy the ForgeKernel harness runtime and configure the autonomous optimization parameters.
- 3Launch the AI agent for iterative optimization, allowing it to automatically generate, execute, and test different kernel versions in a black-box state.
- 4Conduct multiple rounds of iteration and version testing (e.g., 32 kernel versions generated across 42 iterations) to collect and evaluate runtime performance data.
- 5Obtain the final optimized kernel that outperforms the baseline, which can then be applied to operators such as attention and GEMM.
Copy-ready prompt
Referring to the ForgeKernel black-box optimization methodology, for [target performance metric or black-box code block], and without access to source code and documentation, autonomously write and optimize a [optimization target, e.g., CUDA kernel version] that achieves performance gains through a cycle of "generate-execute test-iterate based on benchmark feedback".
#AI Agent#GPU Optimization#Automated Workflow#LLM Practice
Related use cases
Summarized by AI from the original source - refer to the original post for details. Copyright belongs to the original author.