Any further suggestions /optimizations I shall do as in SPARKSQL Hudi table creation, bulk_insert is the only mode available by default i know i can switch to DF API'S but using SQL can we switch to normal insert? Sign up for free to join this conversation on GitHub. Already have an ...
Thanks yes, we are unblocked - we ended up rewriting the entire dataset with bulk insert and sorting it (NB: sorting with df.sort seems faster and allows sorting by a non-record key). After that the number of files touched was much smaller so we didn't run into this (commits were mu...
If you recieve a complaint about an address unjustly getting blocked, place it in one of the MAX_OFFENSES blocks, and the IP will be unblocked the next timeban2failruns in production mode. Working withban2fail There are two primary modes in whichban2failis used: ...