In a C# program I am importing a large text file (300mb) into a MySQL database via the MySqlBulkLoader function of the MySql .Net Connector.
The import takes quite a long time and results in almost 100% disk usage on the Windows 2003 Server it is running on. In an attempt to speed up the import the program nows splits the big file into smaller chunks.
Would it be possible to read the small file chunk (8mb) into memory (ie array) and then pass it to the MySQLBulkLoader as a file?
The bulk loader looks for a filename path:
MySql.Data.MySqlClient.MySqlBulkLoader myBulk = new MySql.Data.MySqlClient.MySqlBulkLoader(connection);
myBulk.Timeout = 10 * 60; /
myBulk.TableName = "some_table";
myBulk.Local = true;
myBulk.LineTerminator = @"\n";
myBulk.FileName = aFile.FullName;
myBulk.FieldTerminator = "";
Memory isn't a file, so the short answer is no. The alternatives are:
System.IO.Path.GetTempFileName()is your friend here, for the name to give the partial file) and passing that filename to MySqlBulkLoader