cache eviction

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

cache eviction

Erik Selin
Hey,

I was wondering how to evict a cached data set so to force it being rebuilt from the underlying hdfs data. After reading the docs I'm still confused how to ensure that intermediate cached result sets are evicted if the underlying hdfs data changes. Are there any access points to force eviction from the application layer?

Thanks!

Erik
Reply | Threaded
Open this post in threaded view
|

Re: cache eviction

Sriram Ramachandrasekaran
RDD suports an unpersist(blocking: Boolean) method. It takes a boolean arg which allows you to either remove the blocks in a blocking or async fashion. 


On Fri, Jan 3, 2014 at 9:49 AM, Erik Selin <[hidden email]> wrote:
Hey,

I was wondering how to evict a cached data set so to force it being rebuilt from the underlying hdfs data. After reading the docs I'm still confused how to ensure that intermediate cached result sets are evicted if the underlying hdfs data changes. Are there any access points to force eviction from the application layer?

Thanks!

Erik



--
It's just about how deep your longing is!