Printing Logs in map-partition

[abby37]

I want to print some logs in transformation mapPartitions  to logs the
internal working of function.
I have used following techniques without any success.
1. System.out.println()
2. System.err.println()
3. Log4j - logger.info
4. Log4j - logger.debug

My code for mapPartitions is similar to  this
<https://github.com/broadinstitute/gatk/blob/master/src/main/java/org/broadinstitute/hellbender/tools/spark/bwa/BwaSparkEngine.java#L103>
.

Is there any way to print logs on console in mapPartitions. Thanks for your
time and helping me in advance.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

[lec ssmi]

the logs  printed in the map function exist  in the worker node, you can access it   directly, or you can  browse through webui. 

abby37 <[hidden email]> 于2020年12月23日周三 下午1:53写道：

I want to print some logs in transformation mapPartitions  to logs the
internal working of function.
I have used following techniques without any success.
1. System.out.println()
2. System.err.println()
3. Log4j - logger.info
4. Log4j - logger.debug

My code for mapPartitions is similar to  this
<https://github.com/broadinstitute/gatk/blob/master/src/main/java/org/broadinstitute/hellbender/tools/spark/bwa/BwaSparkEngine.java#L103>
.

Is there any way to print logs on console in mapPartitions. Thanks for your
time and helping me in advance.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

[abby37]

I have looked into the logs of worker node by going to the logs folder of
spark. I didn't find in logs my print statements from log4j or System.out.

Can you give detailed response regarding that. Much appreciated



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]