spark UI storage tab

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

spark UI storage tab

Amit Sharma-2
Hi , I have few questions as below

1. In the spark ui storage tab is displayed 'storage level',' size in memory' and size on disk, i am not sure it displays RDD ID 16 with memory usage 76 MB not sure why it is not getting 0 once a request for spark streaming is completed. I am caching some RDD inside a method and uncaching it.

2. Similarly on Executor tab it display  'Storage Memory' used and available, is that used means currently in use  or memory used on that executor at some point of time (maximum memory used so far)




Thanks
Amit