Block-sparse GPU kernels
OpenAI releases highly-optimized GPU kernels for block-sparse neural networks. These kernels outperform cuBLAS and cuSPARSE by orders of magnitude depending on sparsity levels. They achieved state-of-the-art results in sentiment analysis and text/image generation.