Kmeans example with floats

classic Classic list List threaded Threaded
4 messages Options
agg
Reply | Threaded
Open this post in threaded view
|

Kmeans example with floats

agg
Hi,

I would like to run the spark example with floats instead of doubles.  When I change this:

  def parseVector(line: String): Vector = {
    return new Vector(line.split(',').map(_.toDouble))
  }

to:

  def parseVector(line: String): Vector = {
    return new Vector(line.split(',').map(_.toFloat))
  }

I get an error, saying it is expecting a double.  Any thoughts?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Kmeans example with floats

sowen
I don't think this is Spark-specific. The first definition doesn't
compile, even if java.util.Vector was intended.

Why not just

def parseVector(line: String) = line.split(',').map(_.toFloat)

it returns an Array[Float], which might be entirely fine where you
want a scala Vector.
--
Sean Owen | Director, Data Science | London


On Mon, Feb 17, 2014 at 7:58 PM, agg <[hidden email]> wrote:

> Hi,
>
> I would like to run the spark example with floats instead of doubles.  When
> I change this:
>
>   def parseVector(line: String): Vector = {
>     return new Vector(line.split(',').map(_.toDouble))
>   }
>
> to:
>
>   def parseVector(line: String): Vector = {
>     return new Vector(line.split(',').map(_.*toFloat*))
>   }
>
> I get an error, saying it is expecting a double.  Any thoughts?
>
> Thanks!
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kmeans-example-with-floats-tp1640.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Kmeans example with floats

Matei Zaharia
Administrator
In reply to this post by agg
The Vector class is defined to work on doubles right now. You’d have to write your own version for floats.

Matei

On Feb 17, 2014, at 11:58 AM, agg <[hidden email]> wrote:

> Hi,
>
> I would like to run the spark example with floats instead of doubles.  When
> I change this:
>
>  def parseVector(line: String): Vector = {
>    return new Vector(line.split(',').map(_.toDouble))
>  }
>
> to:
>
>  def parseVector(line: String): Vector = {
>    return new Vector(line.split(',').map(_.*toFloat*))
>  }
>
> I get an error, saying it is expecting a double.  Any thoughts?
>
> Thanks!
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kmeans-example-with-floats-tp1640.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Kmeans example with floats

MLnick
If you really need floats and don't want to use the MLlib functionality for KMeans, then you can take a look at breeze which has vectors for Double, Float, Int etc https://github.com/dlwh/breeze


On Mon, Feb 17, 2014 at 10:35 PM, Matei Zaharia <[hidden email]> wrote:
The Vector class is defined to work on doubles right now. You’d have to write your own version for floats.

Matei

On Feb 17, 2014, at 11:58 AM, agg <[hidden email]> wrote:

> Hi,
>
> I would like to run the spark example with floats instead of doubles.  When
> I change this:
>
>  def parseVector(line: String): Vector = {
>    return new Vector(line.split(',').map(_.toDouble))
>  }
>
> to:
>
>  def parseVector(line: String): Vector = {
>    return new Vector(line.split(',').map(_.*toFloat*))
>  }
>
> I get an error, saying it is expecting a double.  Any thoughts?
>
> Thanks!
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kmeans-example-with-floats-tp1640.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.