Folder

Checks the contents of a folder for size, age and count. Folder based checks are useful in a number of scenarios:

• Verifying that backups have been uploaded and are the appropiate size
• Checking that logs or other temporary files are being cleaned up
• For batch processes:
• Checking if files are being processed (and/or produced)
• Checking the size of queue processing backlog
• Checking if any error (.err or .log) files have been produced.

folder-check.yaml

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: folder-check
spec:
  interval: 30
  folder:
    - path: /etc/
      name: folder-check-min
      description: Checks if there are at least 10 files in the folder
      minCount: 10

Field	Description	Scheme	Required
path	A local folder path or a remote folder (SMB, SFTP, S3, GCS)	string	Yes
filter	Filter objects out before checking if they are valid	FolderFilter
minCount	The minimum number of files inside the path	int
maxCount	The maximum number of files inside the path, can be used in conjunction with filter.regex to detect error files	int
minAge	The youngest age a file can be	Duration
maxAge	The oldest age a file can be, often used to check for unprocessed files or files that have not been cleaned up	Duration
minSize	The minimum file size, can be used to detect backups that did not upload successfully	Size
maxSize	The maximim file size	Size
*	All other commons field	Common

Duration

Durations are strings with an optional fraction and unit e.g. 300ms, 1.5h or 2h45m. Valid time units are ms, s, m, h.

Size

Sizes are string with a unit suffix e.g. 100 / 100b, 10mb, Valid size units are kb, mb, gb, tb

FolderFilter

Field	Description	Scheme	Required
maxAge	MaxAge the latest object should be younger than defined age	Duration
maxSize	MaxSize of the files inside the searchPath	Size
minAge	MinAge the latest object should be older than defined age	Duration
minSize	MinSize of the files inside the searchPath	Size
regex	Filter files based on regular expression	regex

e.g. to verify that database backups are being performed

postgres-backup-check.yaml

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: folder-check
spec:
  interval: 30
  folder:
    - path: /data/backups
      filter:
        regex: "pg-backups-.*.zip"
      maxAge: 1d # require a daily backup
      minSize: 10mb # the backup should be at least 10mb

Result Variables

The following fields are available in test, display and transform scripts.

Field	Scheme
Oldest	os.FileInfo
Newest	os.FileInfo
MinSize	os.FileInfo
MaxSize	os.FileInfo
SupportsTotalSize (Only true for SMB folders)	bool
SupportsAvailableSize (Only true for SMB folders)	bool
TotalSize	int64
AvailableSize	int64
Files	[]os.FileInfo

FolderFilter

Field	Description	Scheme	Required
minAge		Duration
maxAge		Duration
minSize		Size
maxSize		Size
regex		string