Alex Navarro Alex Navarro - 6 months ago 21
Java Question

Taking photos from a website, and loading them in JavaFX

Right now I'm working on a GUI project, where I'm trying to take photos, found from the URLs I find from inside the source code of a website, and load them into my JavaFx GUI.
For example, I wish for Java to load the website http://www.imdb.com/movies-in-theaters/?ref_=nv_tp_inth_1, and collect all of the "cover photos"/thumbnails that you see as you scroll down the page (no matter the size of the image), and then load them into the GUI view (into an HBox full as a bunch of ImageViews for example).

More in-depth as well, eventually I would like to get it to the point, that the user could click on the image/imageview, and (again for example) it would show show the trailer for the selected movie. (My thinking, is that the trailer link would be found from website, if you clicked through and went to the next page, found the link, went to youtube, and removed all of the content except for the video player necessarily).

In the web-browser that I use, I have access see the page's HTML elements/design, and look through all of the source coding. After just a few twirls, I can easily find the direct URL to the thumbnail/image I'm looking for, and I've found that in javaFX I can load an image into my GUI as a URL, like so:

Image img = new Image("http://website/websiteSubPage/websiteImage");
ImageView imgView = new ImageView(img);


I've also found that the concept of what I'm looking for is called WebScraping... But all of the modules I've looked into and researched so far aren't helping with what I need. The closest module I've found so far, is HtmlUnit. However, HtmlUnit is all about web automation -- And I couldn't find anything in it's documentation on finding a photo, and loading it as a Java Image object, that is callable into an ImageView.

My best guess at the moment, is to have Java load the website in the background, gather the source code, and then I could create a String Manipulator of sorts, that would essentially just find, trim to, and load the URL of every image it finds, and put it into an HBox full of ImageViews.

Ultimately, I feel like my only solution looks something like this:

public HBox listView(){
HBox temp = new HBox();
// Load the website
// Load the source code into a large string.
for (int i=0; i>=<numberOfPhotosPreCalculatedSomeHow>;i++){
Image img = new Image( /*Manipulated string algorithm to find the next image URL*/);
ImageView imgView = new ImageView(img);
ImageView.setOnAction(e -> { /* load the trailer */ }; } // (Lambda)
temp.getChildren().add(ImageView);
}
return temp;
}


However, doing all of this... Makes me feel like I'm doing something horribly wrong, and I need some help.

Thoughts? Is there a module or plugin built specifically for this? Is this possible, or just dumb?

Answer

Found the answer! There's built-in java methods that can allow me to scan in information from a website, and then decipher it as needed.

In my case, here's the code I used:

import java.io.IOException;
import java.net.URL;
import java.util.Scanner;
public class WebReader {
    // Class variable to hold our found URLs :)
    ArrayList<String> listyArray;
    /**
     * @author alexnavarro
     */
    public static void main(String[] args) throws IOException {
        // Gather page & URL data, and read it
        String address = "http://reddit.com";
        URL pageLocation = new URL(address);
        Scanner in = new Scanner(pageLocation.openStream());
        // Initialize an ArrayList to store all of our collected URLs
        listyArray = new ArrayList<String>();
        // Decipher the code line by line
        while(in.hasNext()){
            String line = in.next();
            if (line.contains("href=\"http://")){
                int from = line.indexOf("\"");
                int to = line.lastIndexOf("\"");
                System.out.println(line.subString(from + 1, to);
                listyArray.add(line.subString(from + 1, to);
            }
        }
        // Next, we implement into JavaFx
        launch(args);
    }
    @Override
    public void start(Stage primaryStage){
        primaryStage.setTitle("My loaded photos");
        // Create a place to put our content
        HBox content = new HBox();
        ScrollPane scrollPane = new ScrollPane(content);
        scrollPane.setFitToHeight(true);
        for (int i = 0; i <= listyArray.size() - 1; i++) {
            Image img = new Image(listyArray.get(i));
            ImageView imgView = new ImageView(img);
            content.getChildren().add(imgView);
        } // Launch and sail away!! :)
        Scene s = new Scene(scrollPane, 800, 600);
        primaryStage.setScene(s);
        primaryStage.show();
    }
}

So this was the solution that I was able to find-- I can't believe it took me so long to find a solution, but I hope this helps anybody who is on the same boat that I am. :)