Memory issue - critical Production

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Memory issue - critical Production

dori waldman
This post has NOT been accepted by the mailing list yet.
This post was updated on .
I am getting lots of exceptions  

 java.lang.Exception: Could not compute split, block input-21-1500245456400 not found

I read that its memory issue , spark search for data that should have been in memory but its not there..

I added more nodes to the cluster and also increased memory (I have now 1T of memory) it didnt helped
I saw that I might need to tune memory ( storage / execution) but as I am using spark 2.0 this is not relevant as spark manage memory for me

I tried to change memory to disk persistence , and it didnt improve

Any idea ?  

https://0x0fff.com/spark-memory-management/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Memory issue - critical Production

yncxcw
This post has NOT been accepted by the mailing list yet.
hi,

It's possible that some executors fail during execution.  You can check the logs for each executor and identify the reason that causes the problems. From my perspective, if it is caused by OOM, you can increase the spark.executor.memory for each executor to increase the JVM heap.



Wei

On Mon, Jul 17, 2017 at 12:32 AM, dori waldman [via Apache Spark User List] <[hidden email]> wrote:
I am getting lots of exceptions  

 java.lang.Exception: Could not compute split, block input-21-1500245456400 not found

I read that its memory issue , if search for data that should have been in memory but it cant find it.

I added more node to the cluster and also increased memory (I have now 1T of memory) it didnt helped
I saw that I might need to tune memory ( storage / execution) but as I am using spark 2.0 this is not relevant as spark manage memory for me

I tried to change memory to disk persistence , and it didnt improve

Any idea ?  

https://0x0fff.com/spark-memory-management/


If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/Memory-issue-critical-Production-tp28864.html
To start a new topic under Apache Spark User List, email [hidden email]
To unsubscribe from Apache Spark User List, click here.
NAML

Loading...