ddsprasad ddsprasad - 1 month ago 19
Scala Question

Scala/Java Spark

I need help on developing enterprise scale Java/Scala applications with multiple components/modules involving multiple JARS.

The purpose is, currently the exsisted code is developed as a single component with a single JAR, basically I need some guidance on how to break-up this huge component into multiple Spark/Scala modules with its own JAR, also all the common util functions need to be in the common component like CORE.

The idea behind this refactoring is to call the individual components in parallel depending on the use case and not call each module in sequence.

Are there any tools to make this process (at least partially) automatic or do we have any plug-ins built in in the IntelliJ IDE?

Suggestions or pointers are welcome.

Answer

AFAIK there is no tool which does this.. But you can use Maven Archetype (command line maven option) to generate a multi module project (aggregator-style).

I'm giving an example of maven multi module project. I think there will be same way you can do it for Gradle or other tools adjust your code

Create an archetype from a multi-module project

Creating an archetype for a multi-module project is as simple as creating one for a single-module project.

Just move to the root directory of a multi-module project and call mvn archetype:create-from-project. In the archetype used as an example, some files need to be non filtered. This is achieved by giving the archetype plugin some values in the archetype.filteredExtensions property.

$ mvn archetype:create-from-project -Darchetype.filteredExtensions=java
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO]   archetype
[INFO]   archetype :: business domain model
[INFO]   archetype :: business services
[INFO]   archetype :: persistence layer
[INFO]   archetype :: infrastructure
[INFO]   archetype :: remoting
[INFO]   archetype :: web services
[INFO]   archetype :: web application
[INFO]   archetype :: enterprise application
[INFO] Searching repository for plugin with prefix: 'archetype'.
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype
[INFO]    task-segment: [archetype:create-from-project] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] Preparing archetype:create-from-project
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: business domain model
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: business services
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: persistence layer
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: infrastructure
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: remoting
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: web services
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: web application
[INFO] ------------------------------------------------------------------------
[INFO] [gwt:eclipse {execution: default}]
[INFO] Unpack native libraries required to run hosted browser
[INFO] [gwt:generateAsync {execution: default}]
[INFO] ------------------------------------------------------------------------
[INFO] Building archetype :: enterprise application
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO] Setting property: velocimacro.messages.on => 'false'.
[INFO] Setting property: resource.loader => 'classpath'.
[INFO] Setting property: resource.manager.logwhenfound => 'false'.
[INFO] [archetype:create-from-project]
[INFO] Setting default groupId: com.capgemini.archetype
[INFO] Setting default artifactId: archetype
[INFO] Setting default version: 1.0.0-SNAPSHOT
[INFO] Setting default package: com.capgemini
[INFO] Archetype created in target/generated-sources/archetype
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7 seconds
[INFO] Finished at: Mon Sep 15 19:11:17 CEST 2008
[INFO] Final Memory: 19M/36M
[INFO] ------------------------------------------------------------------------

Also look at