Liturgist Liturgist - 3 months ago 15
PowerShell Question

Changing one field in a CSV file fails using pipes

I need to modify one (1) field in a CSV file. I can do it with separate commands, but the CSV file may be large (GiB+). My understanding is that

Import-Csv
will read the entire file into memory unless it is piped to another process. Is that correct?

Using separate commands on a small file works, but the pipelined command produces no output. What am I missing?

PS C:\src\powershell> $PSVersionTable.PSVersion

Major Minor Build Revision
----- ----- ----- --------
4 0 -1 -1

PS C:\src\powershell> Get-Content .\eximtest.ps1
$infile = ".\eximtest.csv"

"id,name,breed
1,Ruby,cat
2,Ralph,dog
3,Asia,cat" | Out-File $infile

# Non-pipeline approach, reads all of $infile into $csv
$csv = Import-Csv $infile
foreach($row in $csv) { $row.name = $row.name + "-bedoo" }
$csv | Export-Csv ".\eximtest-a.csv" -NoTypeInformation

# Pipeline approach, pipes $infile to next process as read
Import-Csv $infile | `
foreach($_) { $row.name = $row.name + "-bedoo" } | `
Export-Csv ".\eximtest-b.csv" -NoTypeInformation


Running the script produces a correct file (nevermind the quoting). But the pipelined command produces a zero (0) length file.

PS C:\src\powershell> .\eximtest.ps1

PS C:\src\powershell> Get-ChildItem .\eximtest-*.csv


Directory: C:\src\powershell


Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 2016-08-17 14:12 94 eximtest-a.csv
-a--- 2016-08-17 14:12 0 eximtest-b.csv


Many thanks all. Working version.

Import-Csv $infile | `
Foreach-Object {$_.name = $_.name + "-bedoo"; $_} | `
Export-Csv ".\eximtest-b.csv" -NoTypeInformation

Answer

Your pipeline version is mixed up and has no output (you modify, but don't write anything to the output pipeline).

Import-Csv $infile |
    ForEach-Object {
        # This sets the value
        $_.Name = $_.Name + '-bedoo'
        # This is output (post-modification)
        $_
    } |
    Export-Csv ".\eximtest-b.csv" -NoTypeInformation
Comments