reading snappy eventlog files from hdfs using spark

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

reading snappy eventlog files from hdfs using spark

satishl
This post has NOT been accepted by the mailing list yet.
Hi, I am planning to process spark app eventlogs with another spark app. These event logs are saved with snappy compression (extension: .snappy). When i read the file in a new spark app - i get a snappy library not found error. I am confused as to how can spark write eventlog in snappy format without an error, but reading fails with the above error.

Any help in unblocking myself to read snappy eventlog files from hdfs using spark?
Loading...