PySpark Streaming : Accessing the Remote Secured Kafka
Currently, I am using PySpark Streaming (Classic Regular DStream Style and not Structured Streaming). Now, our remote Kafka is secured with Kerberos.
To enable PySpark Streaming to access the secured Kafka, what steps I should perform? Can I pass the principal/keytab and jaas config in the Spark Submit command? What are my
options? Any pointers/links to access the secured Kafka broker from PySpark Streaming will be helpful.