2个孩子的爸爸,兴趣在通用计算系统、AI计算系统和算法From Online Softmax to FlashAttention ,UW CSE 599M Spring 2023: ML for ML Systems这个对online softmax 思想到flashattention思想写的详细,解答了很多疑问,论文有疑问的可以看这个公式推导。recurrence的思想通过形式化很清晰了,有点疑问第二张图的V矩阵写成...
Indentation is significant Every statement in the body must have exactly the same indentation That’s how Python knows where the body ends for i in [3,4,5]: print "Start body" print i print i*i Compare the results of these loops: for f in [30,40,50,60,70]: print f, (f-32)/...