alifirat alifirat - 14 days ago 4
Scala Question

How copy resources files with sbt docker plugin

I'm running an application using docker which crashes because my application cannot access to a file located in the folder

src/main/resources
. Here my file
project/plugins.sbt
:

addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.1.4")


my
build.sbt
file :

organization := """foo"""

name := """bar"""

version := "1.0"

scalaVersion := "2.11.7"

libraryDependencies ++= Seq(
"org.apache.kafka" % "kafka_2.10" % "0.10.0.0",
"org.scalatest" % "scalatest_2.11" % "3.0.0-M16-SNAP1",
"org.scalaz" %% "scalaz-core" % "7.2.5",
"com.typesafe.akka" %% "akka-actor" % "2.4.11",
"com.typesafe.play" % "play-json_2.11" % "2.4.8",
"com.typesafe.scala-logging" %% "scala-logging" % "3.1.0",
"com.typesafe" % "config" % "1.3.0",
"com.amazonaws" % "aws-java-sdk" % "1.11.8",
"ch.qos.logback" % "logback-classic" % "1.1.7",
"com.typesafe.akka" % "akka-testkit_2.11" % "2.4.12"
)

// the default value in sbt is set to true
parallelExecution in Test := false

resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/conigtent/repositories/snapshots"

dependencyOverrides ++= Set(
"com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)

resolvers ++= Seq(
"Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/",
"Sonatype Releases" at "https://oss.sonatype.org/content/repositories/releases/"
)

// http://www.scala-sbt.org/sbt-native-packager/formats/docker.html
enablePlugins(DockerPlugin, JavaAppPackaging)
// enablePlugins(JavaAppPackaging)
packageName in Docker := "foo/bar"


Here the main file :

object FGS_MainApp extends LazyLogging {


val actorSystemName = "SYS"
println("Create actor sys " + actorSystemName)
val system = ActorSystem(actorSystemName)

private val t = new Thread {
override def run() = {
if (system != null) {
Database.closeConnection
system.terminate
logger.info("ACTOR SYSTEM SHUTDOWN")
}
}
}

Runtime.getRuntime.addShutdownHook(t)

def main(args : Array[String]) = {
// TODO : this needs to be remove quickly
val saslPath = getClass.getResource("/sasl.conf").getPath
println("saslPath = " + saslPath)
System.setProperty("java.security.auth.login.config", saslPath)
}
}


Then when I'm running the command :
sbt docker:publishLocal && docker run -it foo/bar:1.0
, I got this error :

Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
No such file or directory


The printed path is :

file:/opt/docker/lib/foo.bar-1.0.jar!/sasl.conf


What did I miss with the configuration in
build.sbt
?

Update



Caused by: org.apache.kafka.common.KafkaException: java.lang.SecurityException: java.io.IOException: Configuration Error:
No such file or directory
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
... 18 more
Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
No such file or directory
at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:137)
at sun.security.provider.ConfigFile.<init>(ConfigFile.java:102)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at javax.security.auth.login.Configuration$2.run(Configuration.java:255)
at javax.security.auth.login.Configuration$2.run(Configuration.java:247)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.Configuration.getConfiguration(Configuration.java:246)
at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61)
at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:46)
at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68)
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
... 21 more
Caused by: java.io.IOException: Configuration Error:
No such file or directory
at sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:335)
at sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:271)
at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:135)
... 35 more


And it shows the line :

private val kafkaConsumer = new KafkaConsumer[String, Array[Byte]](kafkaProps)
where
kafkaProps
is type of
java.util.Properties
. So it cannot find the file in resources folder.

Answer

The problem you're encountering is with how src/main/resources is treated. It's a special folder whose contents will be included in the jar file built by sbt. It's never actually copied as a file into your universal package. That's why the file path that's printed out includes the jar name and a ! - that means the file is inside the zipped jar. It's clear that the kafka library doesn't handle this.

To solve this, you should put the file in a different location, like src/main/conf (optional, but strictly-speaking more correct), then map it into your universal package:

mappings.in(Universal) +=
  ((sourceDirectory.value / "main" / "conf" / "sasl.conf"), "conf/sasl.conf"))

This will now be available as the file conf/sasl.conf within your universal package.

Comments