Aleksey Kiselev Aleksey Kiselev - 21 days ago 7
Java Question

Error on attempt to recieve JSON data from Cassandra DB via Java + sparkSession

I'm trying to read data from Cassandra table via Java + sparkSession, method should return it as a JSON.

Here is my DB:

CREATE TABLE user (
user_id uuid,
email text,
first_name text,
last_name text,
user_password text,
created_date timestamp,
updated_date timestamp,
PRIMARY KEY (user_id)
) WITH comment = 'List of all registered and active users';


and here is a code which should return JSON:

public String getAccountData(UUID userid) throws ClassNotFoundException, SQLException {
SparkSession sparkSession = config.getSparkSession();
//"SELECT user.first_name,user.last_name, user.email FROM chat.user where user.id="+userid+";");

Account account = new Account();
Encoder<Account> accountEncoder = Encoders.bean(Account.class);

return sparkSession
.read()
.format("org.apache.spark.sql.cassandra")
.options(new HashMap<String, String>() {
{
put("keyspace", "chat");
put("table", "user");
}
})
.load()
.select("first_name", "last_name", "email")
.filter("user_id = '" + userid +"'")
.toJSON()
.as(accountEncoder)
.toString();
}


and here is my Account.java file:

package rest.account;

import java.io.Serializable;


public class Account implements Serializable {

private String user_id;
private String first_name;
private String last_name;
private String email;

public Account(){}

public Account(String user_id, String first_name, String last_name, String email){
this.user_id = user_id;
this.first_name = first_name;
this.last_name = last_name;
this.email = email;
}
//------------------------------
public String getId() {
return user_id;
}

public void setId(String user_id) {
this.user_id = user_id;
}

//------------------------------
public String getFirstName() {
return first_name;
}

public void setFirstName(String first_name) {
this.first_name = first_name;
}

//------------------------------
public String getLastName() {
return last_name;
}

public void setLastName(String lastName) {
this.last_name = last_name;
}
//------------------------------
public String getEmail() {
return email;
}

public void setEmail(String email) {
this.email = email;
}
}


And here is the output error:


HTTP Status 500 - org.glassfish.jersey.server.ContainerException:
org.apache.spark.sql.AnalysisException: cannot resolve '
email
' given
input columns: [value];

type Exception report

message org.glassfish.jersey.server.ContainerException:
org.apache.spark.sql.AnalysisException: cannot resolve '
email
' given
input columns: [value];

description The server encountered an internal error that prevented it
from fulfilling this request.

exception

javax.servlet.ServletException:
org.glassfish.jersey.server.ContainerException:
org.apache.spark.sql.AnalysisException: cannot resolve '
email
' given
input columns: [value];
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489)
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:410)
org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:169)
root cause

org.glassfish.jersey.server.ContainerException:
org.apache.spark.sql.AnalysisException: cannot resolve '
email
' given
input columns: [value];
org.glassfish.jersey.servlet.internal.ResponseWriter.rethrow(ResponseWriter.java:278)
org.glassfish.jersey.servlet.internal.ResponseWriter.failure(ResponseWriter.java:260)
org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:509)
org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:334)
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:410)
org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:169)
root cause

org.apache.spark.sql.AnalysisException: cannot resolve '
email
' given
input columns: [value];
org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:77)
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:74)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:301)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:301)
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:69)
org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:300)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:298)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:298)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$5.apply(TreeNode.scala:321)
org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:179)
org.apache.spark.sql.catalyst.trees.TreeNode.transformChildren(TreeNode.scala:319)
org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:298)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:298)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:298)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$5$$anonfun$apply$10.apply(TreeNode.scala:338)
scala.collection.MapLike$MappedValues$$anonfun$iterator$3.apply(MapLike.scala:246)
scala.collection.MapLike$MappedValues$$anonfun$iterator$3.apply(MapLike.scala:246)
scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
scala.collection.Iterator$class.foreach(Iterator.scala:893)
scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
scala.collection.IterableLike$$anon$1.foreach(IterableLike.scala:311)
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
scala.collection.mutable.MapBuilder.$plus$plus$eq(MapBuilder.scala:25)
scala.collection.TraversableViewLike$class.force(TraversableViewLike.scala:88)
scala.collection.IterableLike$$anon$1.force(IterableLike.scala:311)
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$5.apply(TreeNode.scala:346)
org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:179)
org.apache.spark.sql.catalyst.trees.TreeNode.transformChildren(TreeNode.scala:319)
org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:298)
org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionUp$1(QueryPlan.scala:190)
org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$2(QueryPlan.scala:200)
org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$5.apply(QueryPlan.scala:209)
org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:179)
org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionsUp(QueryPlan.scala:209)
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:74)
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:67)
org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:126)
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:67)
org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:58)
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.resolveAndBind(ExpressionEncoder.scala:245)
org.apache.spark.sql.Dataset.(Dataset.scala:210)
org.apache.spark.sql.Dataset.(Dataset.scala:167)
org.apache.spark.sql.Dataset$.apply(Dataset.scala:59)
org.apache.spark.sql.Dataset.as(Dataset.scala:359)
rest.account.AccountManager.getAccountData(AccountManager.java:58)
rest.account.AccountService.getAccountData(AccountService.java:28)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:410)
org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:169)
note The full stack trace of the root cause is available in the Apache
Tomcat/9.0.0.M13 logs.

Apache Tomcat/9.0.0.M13


If I have not this converter to JSON code then method returns, for example:


[first_name: string, last_name: string, email: string]


But not a real values like First Name, Last Name, email@email.email

I will appreciate any help!

Answer

In your getAccountData method try below code.

return sparkSession
    .read()
    .format("org.apache.spark.sql.cassandra")
    .options(new HashMap<String, String>() {
        {
            put("keyspace", "chat");
            put("table", "user");
        }
    })
    .load()
    .select("first_name", "last_name", "email")
    .filter("user_id = '" + userid +"'")
    .toJSON()
    .first();
}