Spark ETL use case

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Spark ETL use case

Can we use spark as a ETL service? Suppose we have data written to our
cassandra data stores and we need to transform and load the same to vertica
for analytics purposes. Since spark is already a very well designed
distributed system, wouldn't it make sense to load data from cass into spark
datasets and then push the same after transformations to vertica? This way
we won't need to implement the ETL service ourselves and make use of already
performant system in place.

Please advice. TIA.

Sent from:

To unsubscribe e-mail: [hidden email]