The reason for this post is that there was no sample syntax so this is was my first change. SPARK_LOCAL_IP=192.168.0.38 export HADOOP_HOME=/home/kub18/hadoop-3.2.1 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native That was accepted by SPARK code.
My second change was : SPARK_LOCAL_IP=192.168.0.38 HADOOP_HOME=/home/kub18/hadoop-3.2.1 LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native that was also accepted.
my third change was SPARK_LOCAL_IP=192.168.0.38 doogie HADOOP_HOME=/home/kub18/hadoop-3.2.1 doogie LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native This change produced a doogie command not found error.
So it is unclear by having the prefix "export" in spark-env.sh as illustrated changes the behaviour of SPARK or NOT.
my suggestion would be to supply example syntax in the spark-env.sh and also give information whether by prefixing "export" to the SPARK environment variable makes a difference or not.