XML Watch Folder Tutorial (Windows)

Automate your ingestion process by sending files to a 'watched' folder. This will then trigger a bespoke import process.

Contents

1. Determine the trigger
2. Configure the process
3. Create the scheduled task
4. Test the watch folder

1. Determine the trigger

   In most cases, the watch folder process will be triggered by an XML sidecar file appearing in the watch folder, which will contain all metadata for the media. The metadata should contain an HTTP(s) link that can be accessed by our storage server IP address. 

   💡Pro tip: we strongly advise that you use XML files to trigger the process, as transfer times for media can cause bugs/delays

   Imagen storage service will need to retrieve the media during the ingestion process, so transferring media to trigger the process will duplicate efforts and significantly reduce efficiency  

   
   
   

    

   💡Pro tip: XML may be transferred via FTPS to trigger the ingest. Check out this guide for a walkthrough tutorial

   Configure the process

    

   The following code excerpt will construct the barebones of a Windows watch folder. Log files or transcripts can be added to aid debugging, and email functions can be added to alert users on failure. 

    

   #########################################
   ### Set the folder to monitor for XML ###
   #########################################
   $watchFolder = [location of your "Monitor" folder]
   
   ##################################################
   ### Set minimum TLS version required by Imagen ###
   ##################################################
   [Net.ServicePointManager]::SecurityProtocol =
   [Net.SecurityProtocolType]::Tls12
   
   ####################################
   ### SET TYPE OF FILES TO MONITOR ###
   ####################################
   $filterCreated = '*.xml'
   
   ###########################################################################
   ### CREATE FILE SYSTEM WATCHERS AND ONLY INCLUDE SPECIFIED FILE TYPE(S) ###
   ###########################################################################
       $fswCreated = New-Object IO.FileSystemWatcher $watchFolder, $filterCreated -Property @{
       IncludeSubdirectories = $false
       NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
   }
   
   #########################################################
   ### GENERATE ACTIONS ACCORDING TO THE "Created" EVENT ###
   #########################################################
   $action=Register-ObjectEvent -InputObject $fswCreated -EventName Created -SourceIdentifier FileCreated -Action{
   
       #######################
       ### INPUT VARIABLES ###
       #######################
       $path = $Event.SourceEventArgs.FullPath
       $name = $Event.SourceEventArgs.Name   
   
       ######################    
       ### IMPORT PROCESS ###
       ######################
       Write-Host "Detected CREATION of $name" -ForegroundColor Green
   
       #################################
       ### SET HEADERS FOR REST CALL ###
       #################################
       $headers = @{"X-Imagen-API-Key"="$apiKey";"Authorization"="Bearer $bearer"}
       $contentType = "text/xml;charset=utf-8"
   
       #########################
       ### attempt REST call ###
       #########################
       try {
           ###############################
           ### REST CALL TO IMPORT XML ### 
           ###############################
           Invoke-RestMethod -uri "https://$mccHost`:83/imcc/v1/import/record/$mappingFile`?workflow=$videoWorkflow" -Headers $headers -ContentType $contentType -Method put -verbose -OutFile $mccResponseFile -InFile $path
           $RESTSuccess="true"
       }
       ###########################
       ### handle REST failure ###
       ###########################
       catch [Exception] {
           $restSuccess = "false"
           Write-Host "Exception = $_"
   
       ##########################################
       ### run final checks on import attempt ###
       ##########################################
       }finally {
           ##########################################
           ### check if REST call was successfull ###
           ##########################################
           if ($RESTSuccess -eq "true") {
       
               #####################################################
               ### Check the XML response to ensure valid import ###
               #####################################################
               $parseResopnse=[xml](Get-Content $mccResponseFile)
   
               ###############################
               ### set XPath to error node ###
               ###############################
               $failureError=$parseResopnse.Response.DatabaseJob.Error.InnerText
             
                   ##################################################
                   ### check to see if there is a failure message ###
                   ##################################################
                   if ($failureError -ne $null) {
                       Write-Error "Import failed because: $failureError"
                       $restSuccess = "false"
                   }#end if import failed
   
                   #######################################################################
                   ### no error has been found so the import must have been successful ###
                   #######################################################################
                   else{
                       Write-Host "Successful Import -- $RESTSuccess"
                       $successIsoDate = Get-Date -Format s
                       $mccResponseContent=[xml](Get-Content $mccResponseFile)
                       $recordID=$mccResponseContent.Response.DatabaseJob.Import.ImagenRecords.ImagenRecord.ID
                       $mediaID=$mccResponseContent.Response.DatabaseJob.Import.ImagenRecords.ImagenRecord.MediaObjects.MediaObject.ID
                       write-host "Imagen Record Number: $recordID Media object number: $mediaID"
                   }#end else successfull import
           }#end if REST success
       }#end finally
   }#end action 

    

   💡Pro tip: the XML file dropped into the watch folder must be in an Imagen XML format that matched your record schema

    

2. Create the scheduled task

   
   

3. Test the watch folder

Alternative Considerations

Serverless code applications

Azure

You may have chosen to create - get from our PS team - an Azure function app, which will allow you to upload XML and media to an Azure storage account to trigger a function

Azure XML watch folder 

AWS

You may have chosen to create - or get from our PS team - an Azure function app, which will allow you to upload XML and media to an Azure storage account to trigger a function

AWS Lambda function