Getting file checksum

The scripting guy has recently posted two blog posts on:

As you can see, I’ve a file less than 200MB from HP (it’s the support pack file):

Get-Item D:\Downloads\psp-9.00.w2k8R2.x64.exe | ft Name,@{l="Size (MB)";e={('{0:N2}' -f ($_.Length/1MB))}} -AutoSize

File size

As there isn’t any built-in function in Windows 2012, I prefered writing a small function to quickly get any hash from a file instead of loading a whole module…

But, I encountered a little problem as I was using native cmdlets to retrieve the raw file:

Get-Content -ReadCount 0 -Path $File  -Encoding byte -ErrorAction Stop

Here’s the memory consumption before running the bad function:
Bad checksum function 01
It almost doubled the memory usage before I got the ‘failed to read file’ error 😦
Bad checksum function 02
Worse, the memory hasn’t be freed up afterward. As you can see I used the following .Net command to return back to normal:

[gc]::collect()

Bad Checksum function 03

So, I’ve corrected my function to use native .Net command instead of native cmdlets.
I used the following instead of the Get-Content cmdlet

[System.IO.File]::ReadAllBytes($File)

I also put a [gc]::collect() in the end block. If there are many files to be checked, it might be better to put it in the process block so that it can free memory after each item handled through the pipeline.

Function Get-CheckSum            
{            
[CmdletBinding()]            
param(            
    [parameter(ValueFromPipeline=$true,Mandatory=$true,Position=0)]            
    [system.string[]]$FilePath = $null,            
            
    [parameter(ValueFromPipeline=$false, Mandatory=$false, Position=1)]            
    [ValidateSet("SHA1","MD5", "SHA256", "SHA384","SHA512")]            
    [system.string[]] ${Type} = "SHA1"            
            
)            
    Begin             
    {            
        Switch ($Type)            
        {            
            SHA1   { $cryptoServiceProvider = New-Object -TypeName System.Security.Cryptography.SHA1CryptoServiceProvider   }            
            MD5    { $cryptoServiceProvider = New-Object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider    }            
            SHA256 { $cryptoServiceProvider = New-Object -TypeName System.Security.Cryptography.SHA256CryptoServiceProvider }            
            SHA384 { $cryptoServiceProvider = New-Object -TypeName System.Security.Cryptography.SHA384CryptoServiceProvider }            
            SHA512 { $cryptoServiceProvider = New-Object -TypeName System.Security.Cryptography.SHA512CryptoServiceProvider }            
        } # end of switch            
    }            
    Process            
    {            
            
        $FilePath | ForEach-Object -Process {            
            $hash = $null            
            $File = $_            
            if (Test-Path $File)            
            {            
                if ((Get-Item -Path $File) -is [System.IO.FileInfo])            
                {            
                    try            
                    {            
                        $hash = $cryptoServiceProvider.ComputeHash(([System.IO.File]::ReadAllBytes($File)))            
                    } catch {            
                        Write-Warning -Message "Failed to read file $File"            
                    }            
                    if ($hash)            
                    {            
                        $result = New-Object System.Text.StringBuilder            
                        foreach ($byte in $hash)            
                        {            
                            [void]  $result.Append($byte.ToString("x2"))            
                        }            
                         ([system.string]$result)            
                    }            
            
                }            
            }            
        }            
    }            
    End             
    {            
        [gc]::collect()            
    }            
}

The above function works now perfectly on the 200 MB file whatever the algorithm requested
Checksum function 01

But it fails on big files :(. At least this time it fails very quickly and without doubling the memory 🙂
Checksum function 02

The above experience raises the following questions:

  • Why was the Get-Content cmdlet failing to read a 200 MB file ?
  • Is there any memory protection mecanism inside Windows 2012 or in powershell V3 that made if failed automatically after the memory usage doubled ?
  • Should we prefer native cmdlet or use .Net ?
  • What are the limits that the cmdlet can handle ? Idem for .Net ?
Advertisements

One thought on “Getting file checksum

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s