Given a natural language in- struction and system observations, the model pre- dicts the goal to complete. Given the goal, the model generates a sequence of actions. The key challenge we address is designing the goal representation. We avoid manually designing a meaning representation, and ...