Is dataframe write blocking? what can be done for fair scheduler?
I have noticed that in fair scheduler setting, if i block on dataframe write
to complete, using AwaitResult, the API call ends up returning, whereas that
is not what I intend to do as it can cause inconsistencies later in the
pipeline. Is there a way to make the dataframe write call blocking?