About pluggable storage roadmap?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

About pluggable storage roadmap?

Liu, Raymond

        I noticed that in spark 1.0 meetup, on 1.1 and beyond roadmap, it mentioned support for pluggable storage strategies. We are also planning on similar things to enable block manager to store data on more storage media.

        So is there any exist plan or design or rough idea on this one already? If yes, can it be shared thus we could see how to fit our plan in.

        Or if not, any idea on what this strategy should cover, other than block manager / shuffle manager ? So we could help to implement this framework?

Best Regards,
Raymond Liu