user41992 user41992 - 2 months ago 14
Vb.net Question

Multi-threading with WebRequest and StreamWriter in VB.NET

I need to make it work in threads for example, thread1 makes call to url with 'order_id=1' and thread2 makes call to url with 'order_id=2' and so on, and the result is then written into the file.

Here's the code:

Public Sub download()
Dim address As String = "http://www.website.com/sales.php&order_id="
Dim FILE_NAME As String = "D:\Folder\Licenses.txt"
Dim index As Integer = 0

Do While index <= 100
If index > 100 Then
Exit Do
End If
Try
Dim request As WebRequest = WebRequest.Create(address & index.ToString)
Dim response As WebResponse = request.GetResponse()
If CType(response, HttpWebResponse).StatusDescription = "OK" Then
Dim dataStream As Stream = response.GetResponseStream()
Dim reader As New StreamReader(dataStream)
Dim responseFromServer As String = reader.ReadToEnd()
If Not File.Exists(FILE_NAME) Then
Using sw As StreamWriter = File.CreateText(FILE_NAME)
sw.WriteLine(responseFromServer)
index += 1
End Using
ElseIf File.Exists(FILE_NAME) Then
Using sw As StreamWriter = File.AppendText(FILE_NAME)
sw.WriteLine(responseFromServer)
index += 1
End Using
End If
End If
Catch ex As Exception
End Try
Loop
End Sub

Answer

A very simple approach would be using Parallel.For(). You just have to make sure that you don't write different things to the output file at the same time. This is solved with the Monitor, which ensures that there will be only one thread in the critical section. Exception handling is omitted:

Dim address As String = "http://www.website.com/sales.php&order_id="
Dim FILE_NAME As String = "D:\Folder\Licenses.txt"

Using fstream As New StreamWriter(FILE_NAME)
    Parallel.For(0, 100, Sub(i As Integer)
                           Dim client As New WebClient
                           Dim content = client.DownloadString(address + i.ToString())
                           Monitor.Enter(fstream)
                           fstream.WriteLine(content)
                           Monitor.Exit(fstream)
                       End Sub)
End Using

This sample could be improved by creating the web clients as thread-local objects. This would avoid creating new clients for every iteration.

Comments