log4j on pyspark

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

log4j on pyspark

Diana Carroll
In a Scala or Java Spark program, I can use log4j to output logging statements.

Is there a way to access log4j in pyspark?  Spark is emitting its own logs that way, so can I use that mechanism to?  If so, how?