第二个转换失败,因为当您将包含 json 字符串的列从数据帧传递到schema_of_json函数 Spark 时,无法确定该列的每一行 json 字符串将计算为相同的架构 要理解为什么所有行具有相同的架构很重要,您必须承认创建schema_of_json函数的主要用例是推断from_json函数的架构。 from_json将 json 字符串转换为struct,基本上是...
I still had to fix some elements with default values in it because Spark was able to infer more correctly and intelligently. For example “0000” was being inferred to long which is correct as per the values but sine it is in double quotes I would expect it as String and this is ho...