XLA shines when the size of the data structures are known in advance, such as in model training. On the other hand, when their dimensions are impossible to determine or certain dynamic slices are used, XLA fails to compile. Modern implementations of text generation are auto-r...
XLA shines when the size of the data structures are known in advance, such as in model training. On the other hand, when their dimensions are impossible to determine or certain dynamic slices are used, XLA fails to compile. Modern implementations of text generation are auto-regres...