Tayla Wilson Tayla Wilson - 6 months ago 27
JSON Question

C# Scrape a website then convert to JSON

This may possibly be a broad question however I'll try to make it as specific as possible.

I have created a store type application which lists Minecraft mods. The way I currently store the information for the store is in a .json file where different variables are declared for my application to read and list them into a listbox.

However, the issue here being obvious is that I would have to manually update this .json file whenever I needed to add new mods or update them. I'm wondering if it's even possible to scrape a website such as spigotmc.org for all of its downloads, and grab the following information from them.

{
"productName": "SupplyDrop",
"category": "Chat",
"version": "4.6",
"minecraftVersions": ["1.8", "1.9"],
"API": ["CraftBukkit", "Spigot"],
"downloadLink": "https://www.spigotmc.org/resources/supplydrop-1-8-1-9.23455/download?version=90759",
"spigotLink": "https://www.spigotmc.org/resources/supplydrop-1-8-1-9.23455/",
"description": "Supply Drop is a very useful plugin for any survival server, especially factions servers. What it does is it randomly drops a chest around the map that contains items in it, for one player to find and receive!",
"imageLink": "https://www.spigotmc.org/data/resource_icons/23/23455.jpg",
"longDescription": "Supply Drop is a very useful plugin for any survival server, especially factions servers. What it does is it randomly drops a chest around the map that contains items in it, for one player to find and receive! This plugin can bring the competitiveness out in your players as they compete for the chest!"
}


This is how individual items are stored in the file.

As said, if possible I'd like to be able to automatically scrape a website such as spigotmc.org for all of the mods and get information and translate it to this system.

Is this even remotely possible?

Answer

If you just need the data, you could use a web Scrapper tool like Outwit , Import.io , Kimono. Or if you need to develop on your own, you could use Regex, HtmlAgilityPack.

you can also have a look at the Articles, which may help you.

Comments