how to inc counter column w Calliope

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

how to inc counter column w Calliope

amoc

Hi

I use Calliope (http://tuplejump.github.io/calliope/show-me-the-code.html) to update some counters into Cassandra.

If instead of counter column I use a text column the updates work, but with the counter type Calliope always fails to prepare cql query.

 

A: Counter of Text type:

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = ? ")

This works for text mode. No surprise here.

 

B: Counter of Counter type:

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = counter + ? ")

 

This gives some byte related error. Here’s the stack trace… some of it:

 

14/02/13 13:13:51 ERROR executor.Executor: Exception in task ID 5

java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:245)

Caused by: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result$execute_prepared_cql3_query_resultStandardScheme.read(Cassandra.java:53653)

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result$execute_prepared_cql3_query_resultStandardScheme.read(Cassandra.java:53630)

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result.read(Cassandra.java:53545)

                at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)

                at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_prepared_cql3_query(Cassandra.java:1820)

                at org.apache.cassandra.thrift.Cassandra$Client.execute_prepared_cql3_query(Cassandra.java:1805)

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:229)

14/02/13 13:13:51 WARN scheduler.TaskSetManager: Lost TID 5 (task 7.0:0)

14/02/13 13:13:51 WARN scheduler.TaskSetManager: Loss was due to java.io.IOException

java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:245)

14/02/13 13:13:51 ERROR scheduler.TaskSetManager: Task 7.0:0 failed 1 times; aborting job

14/02/13 13:13:51 INFO scheduler.TaskSchedulerImpl: Remove TaskSet 7.0 from pool

14/02/13 13:13:52 INFO scheduler.DAGScheduler: Failed to run saveAsNewAPIHadoopFile at CassandraRDDFunctions.scala:185

Exception in thread "main" org.apache.spark.SparkException: Job aborted: Task 7.0:0 failed 1 times (most recent failure: Exception failure: java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4)))

 

 

If I change the query back to one I use for counter columns of text type I get

InvalidRequestException(why:Cannot set the value of counter column counter (counters can only be incremented/decremented, not set))

And this is expected because I should not be allowed to set a counter, only to increment it.

 

 

Extra Notes:

These are the marshallers:

  implicit def keyMarshaller(x: (String, String, String)): CQLRowKeyMap = Map("unique_name" -> x._2, "tuple_minute_timebin -> x._3)

    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1)    <-----these are the values the counter is to be incremented by.

 

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                     .saveWithQuery("UPDATE vsapi.counters set ?WHAT TO PUT HERE? ")

dataMappedToColumnOrder.cql3SaveToCassandra(cas)

 

 

Any ideas how to increment counters with Calliope?

 

Thanks

-A

 

Reply | Threaded
Open this post in threaded view
|

Re: how to inc counter column w Calliope

Rohit Rai

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = counter + ? ")


is the correct thing to use...

The problem here is in the marshaller for the rowValues...  
    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1)

A counter value is a long and can only be incremented with a Long. so the marshaller will be -
    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1.toLong) 

i.e. converting string to long... and that should solve the issues... 

Let us know if you still face trouble.

Will add an example of this to test cases and documentation... Hold me to that :)

Cheers,
Rohit


Founder & CEO, Tuplejump, Inc.
____________________________
The Data Engineering Platform


On Thu, Feb 13, 2014 at 11:53 PM, Adrian Mocanu <[hidden email]> wrote:

Hi

I use Calliope (http://tuplejump.github.io/calliope/show-me-the-code.html) to update some counters into Cassandra.

If instead of counter column I use a text column the updates work, but with the counter type Calliope always fails to prepare cql query.

 

A: Counter of Text type:

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = ? ")

This works for text mode. No surprise here.

 

B: Counter of Counter type:

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = counter + ? ")

 

This gives some byte related error. Here’s the stack trace… some of it:

 

14/02/13 13:13:51 ERROR executor.Executor: Exception in task ID 5

java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:245)

Caused by: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result$execute_prepared_cql3_query_resultStandardScheme.read(Cassandra.java:53653)

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result$execute_prepared_cql3_query_resultStandardScheme.read(Cassandra.java:53630)

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result.read(Cassandra.java:53545)

                at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)

                at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_prepared_cql3_query(Cassandra.java:1820)

                at org.apache.cassandra.thrift.Cassandra$Client.execute_prepared_cql3_query(Cassandra.java:1805)

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:229)

14/02/13 13:13:51 WARN scheduler.TaskSetManager: Lost TID 5 (task 7.0:0)

14/02/13 13:13:51 WARN scheduler.TaskSetManager: Loss was due to java.io.IOException

java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:245)

14/02/13 13:13:51 ERROR scheduler.TaskSetManager: Task 7.0:0 failed 1 times; aborting job

14/02/13 13:13:51 INFO scheduler.TaskSchedulerImpl: Remove TaskSet 7.0 from pool

14/02/13 13:13:52 INFO scheduler.DAGScheduler: Failed to run saveAsNewAPIHadoopFile at CassandraRDDFunctions.scala:185

Exception in thread "main" org.apache.spark.SparkException: Job aborted: Task 7.0:0 failed 1 times (most recent failure: Exception failure: java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4)))

 

 

If I change the query back to one I use for counter columns of text type I get

InvalidRequestException(why:Cannot set the value of counter column counter (counters can only be incremented/decremented, not set))

And this is expected because I should not be allowed to set a counter, only to increment it.

 

 

Extra Notes:

These are the marshallers:

  implicit def keyMarshaller(x: (String, String, String)): CQLRowKeyMap = Map("unique_name" -> x._2, "tuple_minute_timebin -> x._3)

    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1)    <-----these are the values the counter is to be incremented by.

 

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                     .saveWithQuery("UPDATE vsapi.counters set ?WHAT TO PUT HERE? ")

dataMappedToColumnOrder.cql3SaveToCassandra(cas)

 

 

Any ideas how to increment counters with Calliope?

 

Thanks

-A

 


Reply | Threaded
Open this post in threaded view
|

RE: how to inc counter column w Calliope

amoc

That worked! Thanks a lot!

 

PS - Got your other message too

 

From: Rohit Rai [mailto:[hidden email]]
Sent: February-14-14 12:32 AM
To: user
Subject: Re: how to inc counter column w Calliope

 

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = counter + ? ")

 

is the correct thing to use...

 

The problem here is in the marshaller for the rowValues...  

    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1)

 

A counter value is a long and can only be incremented with a Long. so the marshaller will be -

    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1.toLong) 

 

i.e. converting string to long... and that should solve the issues... 

 

Let us know if you still face trouble.

 

Will add an example of this to test cases and documentation... Hold me to that :)

 

Cheers,

Rohit

 


Founder & CEO, Tuplejump, Inc.

____________________________

The Data Engineering Platform

 

On Thu, Feb 13, 2014 at 11:53 PM, Adrian Mocanu <[hidden email]> wrote:

Hi

I use Calliope (http://tuplejump.github.io/calliope/show-me-the-code.html) to update some counters into Cassandra.

If instead of counter column I use a text column the updates work, but with the counter type Calliope always fails to prepare cql query.

 

A: Counter of Text type:

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = ? ")

This works for text mode. No surprise here.

 

B: Counter of Counter type:

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                       .saveWithQuery("UPDATE vsapi.counters set counter = counter + ? ")

 

This gives some byte related error. Here’s the stack trace… some of it:

 

14/02/13 13:13:51 ERROR executor.Executor: Exception in task ID 5

java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:245)

Caused by: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result$execute_prepared_cql3_query_resultStandardScheme.read(Cassandra.java:53653)

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result$execute_prepared_cql3_query_resultStandardScheme.read(Cassandra.java:53630)

                at org.apache.cassandra.thrift.Cassandra$execute_prepared_cql3_query_result.read(Cassandra.java:53545)

                at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)

                at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_prepared_cql3_query(Cassandra.java:1820)

                at org.apache.cassandra.thrift.Cassandra$Client.execute_prepared_cql3_query(Cassandra.java:1805)

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:229)

14/02/13 13:13:51 WARN scheduler.TaskSetManager: Lost TID 5 (task 7.0:0)

14/02/13 13:13:51 WARN scheduler.TaskSetManager: Loss was due to java.io.IOException

java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4))

                at org.apache.cassandra.hadoop.cql3.CqlRecordWriter$RangeClient.run(CqlRecordWriter.java:245)

14/02/13 13:13:51 ERROR scheduler.TaskSetManager: Task 7.0:0 failed 1 times; aborting job

14/02/13 13:13:51 INFO scheduler.TaskSchedulerImpl: Remove TaskSet 7.0 from pool

14/02/13 13:13:52 INFO scheduler.DAGScheduler: Failed to run saveAsNewAPIHadoopFile at CassandraRDDFunctions.scala:185

Exception in thread "main" org.apache.spark.SparkException: Job aborted: Task 7.0:0 failed 1 times (most recent failure: Exception failure: java.io.IOException: InvalidRequestException(why:Expected 8 or 0 byte long (4)))

 

 

If I change the query back to one I use for counter columns of text type I get

InvalidRequestException(why:Cannot set the value of counter column counter (counters can only be incremented/decremented, not set))

And this is expected because I should not be allowed to set a counter, only to increment it.

 

 

Extra Notes:

These are the marshallers:

  implicit def keyMarshaller(x: (String, String, String)): CQLRowKeyMap = Map("unique_name" -> x._2, "tuple_minute_timebin -> x._3)

    implicit def rowMarshaller(x: (String, String, String)): CQLRowValues = List(x._1)    <-----these are the values the counter is to be incremented by.

 

val cas = CasBuilder.cql3.withColumnFamily("vsapi", "counters")

                     .saveWithQuery("UPDATE vsapi.counters set ?WHAT TO PUT HERE? ")

dataMappedToColumnOrder.cql3SaveToCassandra(cas)

 

 

Any ideas how to increment counters with Calliope?

 

Thanks

-A