alwayssummer alwayssummer - 2 months ago 14x
PowerShell Question

PS get-content high memory usage - Is there a more efficient way to filter a file?

I am using get-content to read a largish file (252 MB), but when I use get-content to read it, the powershell process proceeds to consume almost 10 GB of memory. Is this normal behavior?

The array has just shy of 6 million items. It doesn't seem to be remotely in line with the amount of memory being used.

Maybe I'm just going about this the wrong way entirely.

I want to write the line that matches a string and the subsequent line to a new text file.

$mytext = get-content $inpath
$search = "*tacos*"
$myindex = 0..($mytext.count - 1) | Where {$mytext[$_] -like $search}
$outtext = @()
foreach ($i in $myindex){
$outtext = $outtext + $mytext[$i] + $mytext[$i+1]
$outtext | out-file -filepath $outpath


Another option is Select-String:

$search = "tacos"
Get-Content $inpath | Select-String $search -Context 0,1 | Out-File $OutputFile -Append

However, this will produce slightly changed output:

following line

will turn into

> match
  following line

if you want the exact lines from the file:

Get-Content $inpath | Select-String $search -Context 0,1 | foreach {$_.Line | Out-File $OutputFile -Append ; $_.Context.Postcontext |  Out-File $OutputFile -Append}

Btw: Get-Content gets kinda slow once files get really big. Once that happens it might be better to do:

$TMPVar = Get-Content $inpath -Readcount 0
$TMPVar | Select-String....

This will make Get-Content read the entire file at once instead of line by line which is much faster but needs a bit more memory than piping it directly into the next cmdlets.