I am totally confused here, may be because I do not exactly understand this, but why is this required? I have always used SPARK UI and found that more than sufficient. And if you know a bit about how SPARK session works then your performance does have a certain degree of predictability as well.
Sorry about this, but the input may really help.
I may be wrong, but I find such pursuits just a sheer exercise in creating products which in reality are not needed.