Spark 3.0.1 giving warning while running with Java 11

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark 3.0.1 giving warning while running with Java 11

Sachit Murarka
Hi All,

Getting warning while running spark3.0.1 with Java11 .


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.0.1.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release 

 
Kind Regards,
Sachit Murarka
Reply | Threaded
Open this post in threaded view
|

Re: Spark 3.0.1 giving warning while running with Java 11

srowen
You can ignore that. Spark 3.x works with Java 11 but it will generate some warnings that are safe to disregard.

On Thu, Jan 14, 2021 at 11:26 PM Sachit Murarka <[hidden email]> wrote:
Hi All,

Getting warning while running spark3.0.1 with Java11 .


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.0.1.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release 

 
Kind Regards,
Sachit Murarka
Reply | Threaded
Open this post in threaded view
|

Re: Spark 3.0.1 giving warning while running with Java 11

Sachit Murarka
Sure Sean. Thanks for confirmation.

On Fri, 15 Jan 2021, 10:57 Sean Owen, <[hidden email]> wrote:
You can ignore that. Spark 3.x works with Java 11 but it will generate some warnings that are safe to disregard.

On Thu, Jan 14, 2021 at 11:26 PM Sachit Murarka <[hidden email]> wrote:
Hi All,

Getting warning while running spark3.0.1 with Java11 .


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.0.1.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release 

 
Kind Regards,
Sachit Murarka