Fahad Siddiqui Fahad Siddiqui - 4 years ago 181
Scala Question

NoSuchMethodError exception in uber jar

These are dependencies inside is my

pom.xml
file. When I run Driver from my maven project as a Scala application, it works fine. But when I create a
jar-with-dependency
and try to run my project through
spark-submit


spark-submit --class package.signature.Driver --master local[*] /path/to/my/jar-with-dependencies.jar


I get following exception.


Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.http.conn.ssl.SSLConnectionSocketFactory.(Ljavax/net/ssl/SSLContext;[Ljava/lang/String;[Ljava/lang/String;Ljavax/net/ssl/HostnameVerifier;)V
at
com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.(HtmlUnitSSLConnectionSocketFactory.java:125)
at
com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.buildSSLSocketFactory(HtmlUnitSSLConnectionSocketFactory.java:112)
at
com.gargoylesoftware.htmlunit.HttpWebConnection.configureHttpsScheme(HttpWebConnection.java:597)
at
com.gargoylesoftware.htmlunit.HttpWebConnection.createHttpClient(HttpWebConnection.java:532)
at
com.gargoylesoftware.htmlunit.HttpWebConnection.getHttpClientBuilder(HttpWebConnection.java:494)
at
com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:158)
at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1321)
at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1238)
at
com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:346)
at
com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:415)
at
org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:541)
at
org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:530)


Origin of this exception is
.getPageSource
from the code

import org.openqa.selenium.htmlunit.HtmlUnitDriver
import java.util.concurrent.TimeUnit
import scala.concurrent.duration._
import org.apache.spark.Logging

object Selenium extends Logging {
val driver = new HtmlUnitDriver(false)
val implicitWaitTimeout = 30 // seconds
driver.manage.timeouts.implicitlyWait(implicitWaitTimeout, TimeUnit.SECONDS)
def getPageSource(urlPage: String): String = {
driver.get(urlPage)
driver.getPageSource
}
}


What I have tried so far.
Trying to exclude multiple versions of
apache-httpcomponent
. But that didn't work.

Answer Source

It is a transitive dependency problem, it means that you have jars which have this class each having version of that class where this method is not present and by default maven loads that class which it gets first, you can check the class by pressing Ctrl+Shift+T, you can see multiple versions of that class, once you find the class which will not be having that method, in pom.xml for that .jar which contains that file write exclusion for that.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download