The sample contains two snippets -- one which exhibits the stated issue and another which works correctly. My suspicion is that when parsing nested dictionary objects in the Row class, the datatype for all values is being incorrectly set to that of the first key encountered (in the above example "duration") and when the conversion fails, it is being set as None. In the second example in the gist, all values in the nested dictionary are strings and all data is preserved correctly.
I am using version 2.1.0:
>>> print pyspark.__version__ 2.1.0
Please let me know if I am missing something or there some issue in the code sample itself.