High CPU Usages

[Amit Sharma-2]

Hi, I have the spark 2.3.3 version. I am using below configuration

driver memory : 16 GB

Excecuors memory : 12 GB

code per exceutor : 5 cores.

I have spark streaming job which receives certain requests and based on the request we read data from Cassandra and process the request. I noticed that some time CPU usages 659 for my running spark streaming process on driver node. Similar behaviour is noticed on workers nodes also. Can someone help me what could be the root cause .

[Lalwani, Jayesh]

Executors having high CPU usage is a good thing. It shows that executors are processing data. You should really worry if executors are spending lot of time in garbage collection. You don’t want too much garbage collection. You need to check the Spark UI to find out if executors are spending time in GC. Another thing I am worried about is why don’t all executors have higher CPU usage. It tells me that you are underutilizing executors.

 

High CPU usage on the driver is not a good thing. You houldn’t be doing a lot of processing in the driver. However, it can happen if your microbatches are executing really fast. Sometimes Sinks and sources do some processing on the driver which adds overhead to every microbatch. I don’t know if Cassandra’s connector adds this overhead. You might want to try to isolate the problem to find if it’s your code or the connector.

 

From: Amit Sharma <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, December 8, 2020 at 11:03 AM
To: "[hidden email]" <[hidden email]>
Subject: [EXTERNAL] High CPU Usages

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

Hi, I have the spark 2.3.3 version. I am using below configuration

driver memory : 16 GB

Excecuors memory : 12 GB

code per exceutor : 5 cores.

 

I have spark streaming job which receives certain requests and based on the request we read data from Cassandra and process the request. I noticed that some time CPU usages 659 for my running spark streaming process on driver node. Similar behaviour is noticed on workers nodes also. Can someone help me what could be the root cause .