Backup Azure File Storage with Powershell

Azure Files. What is it in the first place? Azure documentation provides the following description: “Azure Files offers fully managed file shares in the cloud that are accessible via the industry standard Server Message Block (SMB) protocol (also known as Common Internet File System or CIFS). Azure File shares can be mounted concurrently by cloud or on-premises deployments of Windows, Linux, and macOS. ”

It is all great, but in my business case I had to find the way how to backup Azure Files, directly mounted to my workstation. And surprisingly I am not able to find the solution for this time being. So what’s next? Powershell!

Decided to write my own solution for backing up files stored in Azure Files and mounted to my pc. For this case I am going to use Azure Storage account with Azure Files and Azure Tables. Azure Files I am going to attach to my local pc and Azure Tables I am going to use to track changes in the files.

Before going through the script you need to have created in Azure:

  1. Resource Group
  2. Storage account with
    1. Azure File Share created and mapped to your pc
    2. Azure Tables created to track changes

In my local configuration Azure Files are going to be mapped as a Z drive.

So how am I going to achieve that? My idea is to scan the files, store it in the temporary Table storage and compare with the primary table storage where I am going to store all the historical scans. In the BackupData Table storage I am going to use PartitionKey  as unique identifier with the following format <date>_<HHmm>. RowKey values are going to store all the hashes of the files to track if the file has been changed, timestamp, Path location and additional column to track if the file was backed up or not. Then compare both table hashes by the file path and file name with the stored hash and of there is a change, copy the file in the backup location. Pretty simple implementation.

 

$ResourceGroup = "Marty-AzureFiles"
$StorageAccountName = "backupsa1"
$StorageAccountKey = (Get-AzureRMStorageAccountKey -ResourceGroupName $ResourceGroup -Name $StorageAccountName).Value[0]
$Ctx = New-AzureStorageContext –StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey 
$tablePermanent = "BackupData"
$tableCurrent = "CurrentTempTable"
$FileStorage = "Z:\temp\Files"
$Backups = "Z:\temp\Backup\"
$TimeScanned = $((Get-Date).ToString('yyyy-MM-dd_HHmm'))
$WasBackedup = $false

$tablePrimary = Get-AzureStorageTable –Name $tablePermanent -Context $Ctx -ErrorAction Ignore
$tableTemp = Get-AzureStorageTable –Name $TableCurrent -Context $Ctx -ErrorAction Ignore

if ($tableTemp -ne $null)
{
    Remove-AzureStorageTable –Name $tableCurrent –Context $Ctx -Force
    Sleep -Seconds 3
}

if ($tablePrimary -eq $null)
{
    $tablePrimary = New-AzureStorageTable –Name $tablePermanent -Context $Ctx
}


$tableTemp = Get-AzureStorageTable –Name $TableCurrent -Context $Ctx -ErrorAction Ignore
sleep -Seconds 30

if ($tableTemp -eq $null)
{
    $tableTemp = New-AzureStorageTable –Name $tableCurrent -Context $Ctx
}

function InsertRow($tabletempo, [String]$partitionKey, [String]$rowKey, [String]$Path)
{
    $entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $partitionKey, $rowKey
    $entity.Properties.Add("Path", $Path)
    $result = $tabletempo.CloudTable.Execute([Microsoft.WindowsAzure.Storage.Table.TableOperation]::Insert($entity))
}

function InsertBackupData($BackupTable, [String]$partitionKey, [String]$rowKey, [String]$Path, [boolean]$WasBackedup)
{
    #PartitionKey = Date, RowKey = Path
    $entity = New-Object "Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity" $partitionKey, $rowKey
    $entity.Properties.Add("Path", $Path)
    $entity.Properties.Add("WasBackedup", $WasBackedup)
    $result = $BackupTable.CloudTable.Execute([Microsoft.WindowsAzure.Storage.Table.TableOperation]::Insert($entity))
}

Function Get-StringHash([String] $String, $HashName = "MD5") 
{ 
    $StringBuilder = New-Object System.Text.StringBuilder 
    [System.Security.Cryptography.HashAlgorithm]::Create($HashName).ComputeHash([System.Text.Encoding]::UTF8.GetBytes($String))| % { 
        [Void]$StringBuilder.Append($_.ToString("x2")) 
    } 
    $StringBuilder.ToString() 
}

$CurrentFileHashes = Get-Childitem $FileStorage -Recurse | Get-FileHash -Algorithm MD5

Foreach ($file in $CurrentFileHashes )
{
    $FilePathHash = Get-StringHash $file.Path 
    $tempRowKey = $file.Hash, $FilePathHash -join "_"
    # Write-Output $FilePathHash $file.hash $file.path $OriginalRowKey
    InsertRow $tableTemp $TimeScanned $tempRowKey $file.Path 
}

$backupFolderPath = $Backups + $TimeScanned
$newdir = New-Item $backupFolderPath -ItemType Directory -Force

######################################################

$queryLastRecordsFromBackupTable = New-Object Microsoft.WindowsAzure.Storage.Table.TableQuery
#Define columns to select.
$list = New-Object System.Collections.Generic.List[string]
$list.Add("PartitionKey")
$list.Add("RowKey")
$list.Add("Path")
$list.Add("WasBackedup")
$queryLastRecordsFromBackupTable.SelectColumns = $list

#Execute the query.
$entities = $tablePrimary.CloudTable.ExecuteQuery($queryLastRecordsFromBackupTable)

#Display entity properties with the table format.
$LastScannedDate = $entities | Select-Object PartitionKey -last 1

#to add: -and backedup=false
$LastScannedRecords = $entities | Where-Object {$_.PartitionKey -eq $LastScannedDate.PartitionKey} | `
    Select-Object PartitionKey, Rowkey, @{ Label = "Path"; Expression = {$_.Properties["Path"].StringValue}}


If ($LastScannedDate -eq $null)
{
    Foreach ($entry in $CurrentFileHashes )
    {
        $WasBackedup = $true
                        
        $FilePathHash = Get-StringHash $entry.Path 
        $prodRowKey = $entry.Hash, $FilePathHash -join "_"
                
        InsertBackupData $tablePrimary $TimeScanned $prodRowKey $entry.Path  $WasBackedup 
        $dir = $entry.Path
        $dir = $dir.substring(2, $dir.length - 2)
        $dir = Split-Path -Path $dir
        $dest = $Backups + $TimeScanned + $dir + "\"
        New-Item $dest -ItemType Directory -Force
        Copy-Item -Path $entry.Path -Destination $dest
    }
}
Else
{
    Foreach ($entry2 in $CurrentFileHashes )
    {
        $hasFound = $false;

        $fph = Get-StringHash $entry2.Path 
        $prk = $entry2.Hash, $fph -join "_"

        Foreach ($lastresults in $LastScannedRecords)
        {


            if ($prk -eq $lastresults.rowkey)
            {
                $hasFound = $true;
             
            }
        }  
        if ($hasFound)
        {

            $fph = Get-StringHash $entry2.Path 
            $prk = $entry2.Hash, $fph -join "_"

            #Write-Output "sutampantys failai: " $entry2.path
            #Write output with the files which matches and dont need to be backed up
            $DontBackup = $false
            #$rowKey3 = $entry2.Path.Replace('\','*')

            InsertBackupData $tablePrimary $TimeScanned $prk $entry2.Path  $DontBackup

        }
        Else
        {
            $WasBackedup = $true
            $fph = Get-StringHash $entry2.Path 
            $prk = $entry2.Hash, $fph -join "_"
       
            InsertBackupData $tablePrimary $TimeScanned $prk $entry2.Path  $WasBackedup
            $dir = $entry2.Path
            $dir = $dir.substring(2, $dir.length - 2)
            $dir = Split-Path -Path $dir
            $dest = $Backups + $TimeScanned + $dir + "\"
            New-Item $dest -ItemType Directory -Force
            Copy-Item -Path $entry2.Path -Destination $dest
        }
    }
}

To automate it you can use it as an Azure runbook, or even schedule the script to run on the Windows Task Manager.

One comment

Leave a Reply

Your email address will not be published.