python script or pipeline step. It shares the common computing context and most of the cases you can just directly convert the Spark Dataframe to Pandas and Dask Dataframe without persisting first to an interme
fromfastscriptimport*@call_parsedefmain(msg:Param("The message",str),upper:Param("Convert to uppercase?",bool_arg)=False):print(msg.upper()ifupperelsemsg) When you run this script, you'll see: $ python examples/test_fastscript.py usage: test_fastscript.py [-h] [--upper UPPER] msg ...
The-xand-Xoptions take acommand templateas a series of arguments (instead of a single string). If you want to add additional options tofdafter the command template, you can terminate it with a\;. The syntax for generating commands is similar to that ofGNU Parallel: ...