But this is not something I am desiring. Apart from documentation, I am following a book by Francois Garillot, Gerard Maas titled as Stream Processing with Apache Spark and found these ways where I can use encoders to provide schema for scala development. Is there any other way that I can use to perform this. My logs are pipe | separated records. My log file records everything in string from source nodes which I can not change.
How do I set up schema so that I can read data from log files with specified datatypes of the fields? If spark can not change a String to Integer, what is the workaround?