A few tips and tricks for handling JSON data with Powershell.
Convert from/to (import/export)
Powershell offers functions to import/export JSON data structures from/to Powershell data structures:
ConvertFrom-JsonConvertTo-Json.
# 1. Get JSON data from a file or some other source:
$file = Get-Content -Path "data.json" -Raw
$json = @"
<# For example copy 'n' paste the JSON sample data from the page linked above #>
"@
# 2. Convert JSON to a PowerShell data structure (PSCustomObject):
$file_data = ConvertFrom-Json -InputObject $file
$data = ConvertFrom-Json -InputObject $json
# 3. Display from that data strcuture a value, modify it, then show the changend value:
# Example A:
$data.'Key 7'[3].'Entry A'
$data.'Key 7'[3].'Entry A' = 1024
$data.'Key 7'[3].'Entry A'
# Example B:
($data.'Product Information'[3].'Products'[2].Details | ? { $_.'Name' -eq 'Bar'}).Version
($data.'Product Information'[3].'Products'[2].Details | ? { $_.'Name' -eq 'Bar'}).Version = 2
($data.'Product Information'[3].'Products'[2].Details | ? { $_.'Name' -eq 'Bar'}).Version
# 4. Convert the data back to proper JSON and save it to a new file:
$new_data = ConvertTo-Json -InputObject $data -Depth 100
Set-Content -Encoding utf8 -Value $new_data -Path outfile.json
When using ConvertTo-Json, the parameter -Depth may be important:
Specifies how many levels of contained objects are included in the JSON representation.
The value can be any number from 0 to 100.
The default value is 2.
Enumerate (loop over) items
So, if you need to enumerate all values of, for something like a (nested) hashtable, you can use GetEnumerator():
> $ht = @{ "Clients" = @{ "Client 1" = @{ "Domain" = "example.net"; "Foo" = "Bar"; "Data" = @(1, 2, 3) } } }
> $ht.Clients.GetEnumerator() | select -expand Name
Client 1
But unfortunately, that isn’t available for a PSCustomObject from a JSON conversion: ðŸ˜
> $j = ConvertFrom-Json -InputObject (Get-Content -Raw -Path .\cfg.json)
> $j.Clients.GetEnumerator()
InvalidOperation: Method invocation failed because [System.Management.Automation.PSCustomObject]
does not contain a method named 'GetEnumerator'.
Instead, one has to use the PSObject Properties (present on all objects; Source): 😊
> $j.Clients.PSObject.Properties | select Name, Value
Name Value
---- -----
Client 1 @{Domain=client1.example.net; Foo=Bar; Data=System.Object[]}
Client 2 @{Domain=client2.example.net; Foo=Bar; Data=System.Object[]}
Client 3 @{Domain=client3.example.net; Foo=Bar; Data=System.Object[]}
Combine multiple datasets to one (v1)
This approach comes from a situation in my day job:
Putting all the required configuration data into one big JSON file seemed not ideal: It would become pretty unwieldy pretty fast, especially since the file will be maintained by hand. So my thinking was to split different facets into separate files, which would then (in an initialization step) be combined into a global variable.
The first file, cfg.json, contains the only elementary settings for all the clients:
{
"Clients":
{
"Client 1":
{
"Domain": "client1.example.net",
"Foo": "Bar",
"Data": ["a", "b", "c"]
},
"Client 2":
{
"Domain": "client2.example.net",
"Foo": "Bar",
"Data": [1, 2, 3]
},
"Client 3":
{
"Domain": "client3.example.net",
"Foo": "Bar",
"Data": ["x", "y", "z"]
}
}
}
The other filem, cfgRoles.json, holds only the role definitions (Roles).
And in this example also just for only one of the clients (Client 1).
Pay attention that the name of the structure (enclosing scope(s) (Clients) and the actual names (Client 1)) are the same in both JSON files! Otherwise the entries can’t be assigned correctly.
{
"Clients":
{
"Client 1":
{
"Roles":
{
"Role 1":
{
"ADGroups":
[
"Grp-R1-ABC",
"Grp-R1-XYZ"
]
},
"Role 2":
{
"ADGroups": ["Grp-R2-ABC", "Grp-R2-XYZ"]
}
}
}
}
}
Then, in some setup step, we read both JSON datasets into two distinct variables and then combine those (if matching) into a new global variable, which will hold all configuration items:
$base = ConvertFrom-Json -InputObject (Get-Content -Raw -Path cfg.json)
$roles = ConvertFrom-Json -InputObject (Get-Content -Raw -Path cfgRoles.json)
# Using the base data as a starting point for the final result:
new-variable -Name CFG -Visibility Public -Scope Global -Value $base
# Loop over all clients and add the roles (if available for that client) as a new NoteProperty:
$base.Clients.PSObject.Properties | % {
try { Add-Member -InputObject $CFG.Clients.($_.Name) `
-MemberType NoteProperty `
-Name "Roles" `
-Value $roles.Clients.($_.Name).Roles
}
catch {}
}
# Example/Test of the end result:
try
{
$CFG.Clients.'Client 1' | fl
$CFG.Clients.'Client 1'.Roles.'Role 2'.ADGroups
}
catch { write-error $_.Exception.Message }
Combine multiple JSON files to one Powershell variable (v2)
This is another approach to combine multiple similar JSON structures (collected from multiple JSON files)
into one global Powershell variable, which one can then traverse in the usual way,
e.g. $global:SingleVar.'Area 1'.'SubArea 1.2'.'Some Key'
$Files = Get-ChildItem -Filter *.json | % { get-content $_ | ConvertFrom-Json }
$var = @{}
ForEach ($File in $Files)
{
ForEach ($i in ($File.psobject.properties | ? { $_.MemberType -eq 'NoteProperty' }))
{
if ([bool] $i.Value)
{
if ($var.ContainsKey($i.Name)) {}
else { $var[$i.Name] += $i.Value }
}
else
{
if ($var.ContainsKey($i.Name)) {}
else { $var.Add($i.Name, @{}) }
}
}
}
# --------------------
Remove-Variable -name SingleVar -scope global
New-Variable -name SingleVar -scope global -value $var
"`r`n-------- `$global:SingleVar --------"
$global:SingleVar
Compare Hash Values of a JSON file and a JSON data structure in memory
I needed to check whether the content of a newly generated value (in memory!) of a JSON data structure
would be different to the value of the current JSON file:
If yes: Overwrite the JSON file; if not: Don’t do anything.
That should be doable (easily) by comparing the hash values of the content of both items — but: Because only one of the items is a file, things are a bit more complicated:
The layout of the file’s content has an influence on the hash value; for example:
Are tabs or spaces used? How many spaces? Will the curly brackets be put on a new line? And so on…
And unlike the data structure that I produce (e.g. as a hashtable or a PSCustomObject),
I’ll have less less/no influence on the format and layout of the JSON file:
It may be created manually or by ConvertTo-JSON or by a completely different tool;
and any of these methods could use a different style.
Luckily, ConvertTo-JSON provides the -Compress parameter, which “omits white space and indented formatting in the output string”.
So, for both cases:
- Get the data (from the file and from the in-memory data structure)
- Normalize the data sets
- Determine and compare the hash values of both data sets
$Algorithm = 'SHA256'
# ---- Content of the JSON file: ----
# Read the file's content and get rid of any superfluous formatting and layout stuff with '-Compress':
$DataA = Get-Content -Path .\foo.json -Encoding UTF8 | ConvertFrom-Json | ConvertTo-JSON -Depth 100 -Compress
$StreamA = [System.IO.MemoryStream]::new([System.Text.Encoding]::UTF8.GetBytes($DataA))
$HashA = Get-FileHash -InputStream $StreamA -Algorithm $Algorithm
write-host ("{0} Hash of file : {1}" -f $Algorithm, $HashA.hash)
# ---- Data structure for JSON in memory: ----
$DataB = @{
Foo = @{
Bar = "123"
Baz = "ABC"
}
}
# Like with the file data: Use '-Compress' to normalize the value for comparison:
$DataB = $DataB | ConvertTo-JSON -Depth 100 -Compress
$StreamB = [System.IO.MemoryStream]::new([System.Text.Encoding]::UTF8.GetBytes($DataB))
$HashB = Get-FileHash -InputStream $StreamB -Algorithm $Algorithm
write-host ("{0} Hash of string: {1}" -f $Algorithm, $HashB.hash)
# ---- Comparing both hash values: ----
if ($HashA.hash -eq $HashB.hash) { write-host "[V] Hashes are equal" -ForegroundColor Green }
else { write-host "[X] Hashes are not equal" -ForegroundColor Red }
Film & Television (58)
How To (73)
Journal (18)
Miscellaneous (4)
News & Announcements (21)
On Software (12)
Powershell (1)
Projects (26)