how to disable replace HDFS checkpoint location in structured streaming in spark3.0.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

how to disable replace HDFS checkpoint location in structured streaming in spark3.0.1

lec ssmi
I have written a demo using spark3.0.0, and the location where the checkpoint file  is saved has been explicitly specified   like     
    stream.option("checkpointLocation","file:///C:\\Users\\Administrator\\Desktop\\test")
But the app still throws an   exception about the HDFS file system. 
Is it not possible to specify the local file system as a checkpoint location now?
Reply | Threaded
Open this post in threaded view
|

Re: how to disable replace HDFS checkpoint location in structured streaming in spark3.0.1

lec ssmi
sorry, the mail title  is a little problematic. "How to disable or replace ......"

lec ssmi <[hidden email]> 于2020年10月14日周三 上午9:27写道:
I have written a demo using spark3.0.0, and the location where the checkpoint file  is saved has been explicitly specified   like     
    stream.option("checkpointLocation","file:///C:\\Users\\Administrator\\Desktop\\test")
But the app still throws an   exception about the HDFS file system. 
Is it not possible to specify the local file system as a checkpoint location now?