Rules and retention #

owncloud.online has two kinds of rules that work independently of each other. The file firewall (app file_firewall) checks every incoming request and rejects it before anything happens. The Workflow app (workflow) does not interfere with access; it assigns tags automatically on upload and later deletes files on a schedule when they carry a tag with a retention period. The oco_expire app deletes files by age alone, without any tags.

The two layers compared #

Install both apps (after entering the API key) and check the state:

sudo -u www-data php8.4 occ market:install file_firewall workflow
sudo -u www-data php8.4 occ app:list

If the apps are already present, occ app:enable file_firewall and occ app:enable workflow turn them on.

The settings are under Settings → Admin → Security (section BW-Tech File Firewall) and Settings → Admin → Workflows & Tags (section Workflow).

File firewall: reject access #

Structure of a rule #

A rule is a rule group with a name, the switches Enabled and Test mode (log only), and one or more conditions. Within a group, all conditions must match (shown as AND between the lines in the user interface). If a group matches, the request is blocked and evaluation ends; further groups are not checked.

In the user interface, Add Group creates a new rule group and Add Condition adds another line. Changes only take effect after Save Rules. Until then, deleting a group or a condition also only removes it from the editor.

The firewall answers blocked WebDAV requests with HTTP 403 and the message Blocked by BW-Tech File Firewall: <rule group name>. The web interface shows this message when an upload is blocked. So choose names that users may see.

Four things are never blocked:

• Requests from administrators.
• Everything that runs on the command line (occ, cron.php). The firewall does not evaluate any rules there.
• Requests without a logged-in user, unless they are for a public link. Public links are evaluated.
• Requests for which the evaluation itself throws an error. In that case the firewall lets the request through (fail open).

Disabled groups are skipped entirely.

Available conditions and operators #

In a user_agent pattern, * may only appear at the start and/or the end: *BadBot* looks for the string anywhere, mirall/* requires it at the start, *Firefox at the end, and a pattern without * requires an exact match. The comparison is case-insensitive.

For regex you also choose a target field: user_group, user_agent, ip_range_ipv4, ip_range_ipv6, request_url, or file_mimetype. If the pattern starts with /, #, ~, or @ and is a complete expression with delimiters (such as /secret/i), the app uses it unchanged. Otherwise the app wraps it itself, preferably as #…#i, as ~…~i if the pattern contains a #, and as @…@i otherwise, and compares case-insensitively. A path such as /dav/files/alice/ is therefore treated as a plain pattern.

Device types are android, ios, desktop, and other. If your own client identifiers are configured under the system key firewall.branded_clients, the collective values all_branded and all_non_branded are added, plus android_branded, ios_branded, or desktop_branded per platform, but only for platforms that actually have an identifier configured there.

Request types #

The public type takes precedence: an upload through a public link counts as public, not as upload. file_upload and webdav_any apply regardless of that. Requests are classified by the entry script (index.php, remote.php, public.php, OCS) and the start of the path after it, not by strings anywhere in the address.

An api rule does not block OCS calls themselves. It only affects file operations that reach the storage layer during such a call; creating a share via OCS, for example, is not blocked.

Where requests are checked #

The firewall hooks into the request flow in four places:

1. A Sabre DAV plugin runs after authentication for every WebDAV method and answers matches with a clean HTTP 403.
2. Downloads from the web interface (ZIP for folders and multiple selections, single file) are also rejected with HTTP 403.
3. A storage wrapper catches access that does not go through WebDAV. It applies to write operations (mkdir, rmdir, fopen in write mode, file_put_contents, unlink, rename, copy, touch) and to permission checks. Since version 2.10.3 it also blocks reads of file content (files, versions, trash bin, cached previews) in GET requests; these then count as download, or as public on public links. Blocked reads end with the error of the reading app, such as HTTP 404 for a preview or 400 in the text editor, not with the firewall message. The wrapper stays out of WebDAV requests, because it would run too early there and turn an intended block into a 503 error.
4. For public links, event listeners are added: share.linkaccess for the files_sharing pages and the dav.public.*.before events for WebDAV via public.php.

Rules on the command line #

# Show all rule groups with log level and hit counters
sudo -u www-data php8.4 occ firewall:list

# Machine-readable
sudo -u www-data php8.4 occ firewall:list --output=json
sudo -u www-data php8.4 occ firewall:list --output=json_pretty

Create a group. Conditions are passed as type:operator:value; several conditions are combined with AND:

sudo -u www-data php8.4 occ firewall:add-rule "Block external networks" \
  -c "ip_range_ipv4:is_not:10.0.0.0/8" \
  -c "request_type:is:upload"

The group can be created disabled or in test mode right away:

sudo -u www-data php8.4 occ firewall:add-rule "Watch only" \
  -c "user_agent:matches:*BadBot*" --test-mode

sudo -u www-data php8.4 occ firewall:add-rule "Not live yet" \
  -c "user_group:is:contractors" --disabled

Time windows can be given in the short form HH:MM-HH:MM, optionally with @timezone, or as a JSON value with start and end, optionally with timezone. The short form works since version 2.10.3; older versions stored it as a string without any effect.

sudo -u www-data php8.4 occ firewall:add-rule "Outside office hours" \
  -c "time:is:18:00-06:00@Europe/Berlin"

# the same as JSON
sudo -u www-data php8.4 occ firewall:add-rule "Outside office hours" \
  -c 'time:is:{"start":"18:00","end":"06:00","timezone":"Europe/Berlin"}'

For regex, a | separates the pattern from the target field:

sudo -u www-data php8.4 occ firewall:add-rule "Bot identifiers" \
  -c "regex:is:/bot/i|user_agent"

Delete, back up, and import:

# The index is 0-based: the first group has index 0
sudo -u www-data php8.4 occ firewall:delete-rule 0

# Backup (without a file argument it goes to stdout)
sudo -u www-data php8.4 occ firewall:export /var/backups/firewall.json

# Append or replace completely
sudo -u www-data php8.4 occ firewall:import /var/backups/firewall.json
sudo -u www-data php8.4 occ firewall:import /var/backups/firewall.json --replace
sudo -u www-data php8.4 occ firewall:import - --replace < /var/backups/firewall.json

Besides the rules, the export contains the log level; the import sets it too. If the file has no log level, the current one stays.

Test mode #

Each rule group has its own Test mode (log only) switch. When a group in test mode matches, three things happen:

• The request is not blocked.
• The match is counted as a test hit.
• The match is logged, regardless of the configured log level, because that is the purpose of test mode. The message reads BW-Tech File Firewall TEST MODE would have blocked request: … and names the user, IP, method, URL, rule name, and the evaluated conditions.

Evaluation then continues, so several test groups can match in one pass. In test mode you can therefore check a new rule against real requests before you make it live.

Hit counters #

For each rule group, the firewall keeps the number of real hits, the number of test hits, and the time of the last hit. The timestamp is written in UTC and shown that way (format Y-m-d H:i:s UTC), regardless of the time zone used by the server or the rule.

A group counts at most once per request, even if several checkpoints trigger the evaluation. The counters are shown in the admin settings and in occ firewall:list; the Reset statistics button resets them.

The counters are tied to the internal ID of the rule group. On save, counter values for groups that no longer exist are discarded.

Logging #

The log level applies to the whole app and has three values:

Change it in the user interface (Logging) or directly:

sudo -u www-data php8.4 occ config:app:set file_firewall log_level --value=1
sudo -u www-data php8.4 occ config:app:get file_firewall log_level

Level 2 writes one line per request and is only meant for troubleshooting. Where the messages end up is described in Server log and error messages.

Time zones #

The Request Time condition has its own time zone selection. If it stays on Server time, the rule uses the time zone of the PHP process, which owncloud.online sets to UTC. For office hours, choose an IANA identifier such as Europe/Berlin; the window is then evaluated in that zone, and daylight saving time is handled automatically. An unknown identifier is dropped on save, and the evaluation falls back to server time.

If the end is before the start, or both are equal, the window spans midnight: 22:00–06:00 matches in the evening and in the morning.

Workflow: assign tags automatically #

A workflow consists of a name, any number of conditions, and an action. All conditions must match. The available action is Add tags: the listed tags are assigned to the file as soon as it is created. Later changes to an existing file do not trigger a workflow. The tags must exist when the workflow is saved; otherwise the app rejects it ("The following Autotagging tags do not exist: …").

If more than one workflow matches, all of them run. There is no "first match wins" as in the firewall.

Workflows and retention periods are under Settings → Admin → Workflows & Tags. + Add new workflow creates a workflow.

Available conditions and operators #

The selection list in the admin settings is built from exactly this list. The app also has a regex condition with the target fields cidr, subnet, requesturl, useragent, and usergroup. There is no entry for it in the user interface, so it can only be used programmatically.

The request type offers only Public Share Link, WebDAV, and Other. Device types are android, ios, desktop, and other, plus the branded values if the system key workflow.branded_clients is set.

The systemtag condition checks the file and its parent folders. Since version 1.1.3, a condition whose tag no longer exists matches neither with "is" nor with "is not", and the log shows a warning. Older versions treated a deleted tag as a match and then tagged every new file.

Time zone of the workflow condition #

Unlike the file firewall, there is no time zone selection here. The Request time condition always uses UTC; the user interface converts the input to the browser's local time. Here, too, a window whose end is before its start spans midnight.

Limits of auto-tagging #

• The only built-in action is setting tags. Other types can only be added with a custom extension through the app's event interface.
• Shares (shared storages) are deliberately not included. An upload into a share does not trigger a workflow for the recipient.
• There is neither a test mode nor a hit counter. A workflow takes effect immediately after saving.
• For the File size condition, the check fails if the size of the running transfer cannot be determined. The workflow then does not match.

Retention #

How a retention period is created #

Under Settings → Admin → Workflows & Tags, the Retention periods section shows the text "Delete files tagged with the following tags after the given time:". + Add new rule creates a retention period. It consists of exactly three values: a tag, a number, and a unit (Days, Weeks, Months, Years). New retention periods are limited to 100 years.

Each tag can have exactly one retention period; a second entry for the same tag is rejected. When a tag is deleted, the app automatically removes the corresponding retention period and background job as well.

How the retention period is enforced #

Retention runs only in background jobs. Without a working cron, nothing happens, see Background jobs (cron).

1. A management job of the app checks every hour (with the default tagbased engine, see below) whether a worker job exists for each retention period, and adds missing ones.
2. Each tag with a retention period has its own worker job, which runs every 24 hours.
3. The worker job computes the cutoff date (now minus the retention period) and fetches the tagged objects in batches of ten. From a tagged folder it walks down the tree: if the folder itself is not due yet, its children are checked.
4. By default, the file's modification time is what counts.
5. The job deletes due files and folders on behalf of their owner. If the trash bin is active, they end up there and can be restored until the trash bin expires them. The job never deletes mount points such as shares and external storages, and it does not descend into received shares.

Optionally, the upload time can be used instead of the modification time:

sudo -u www-data php8.4 occ config:system:set \
  workflow.tag-based-retention.use-upload-time --value=true --type=boolean

This switch only works for files that have an upload time stored at all. If it is missing, the file is not deleted, and the log shows the warning No upload time stored for file id …. Since version 1.1.4 the app records the upload time of new files itself if this switch is set:

sudo -u www-data php8.4 occ config:app:set workflow record_upload_time --value yes

Without it, new files only get an upload time if another app writes it; oco_expire (see below) uses the same value.

The two engines #

A system key controls which engine runs retention:

sudo -u www-data php8.4 occ config:system:get workflow.retention_engine
sudo -u www-data php8.4 occ config:system:set workflow.retention_engine \
  --value=tagbased

With an unknown value, nothing is processed; the app writes a critical message to the log that names the allowed values.

Only with the userbased engine does the user interface also show the option "Delete empty folders that have not been modified for … days". It is stored in the app configuration:

sudo -u www-data php8.4 occ config:app:get workflow folder_retention
sudo -u www-data php8.4 occ config:app:get workflow folder_retention_period

folder_retention is the switch (0/1), folder_retention_period the number of days; without an entry, 7 applies.

What retention cannot do today #

• Delete only. There is no archive stage, no move to other storage, and no intermediate state. A due file is deleted; only the trash bin keeps it for a while.
• No advance warning. The app sends no emails; affected users only learn about the deletion from the activity stream, where it is recorded as an automated action.
• No dry run. There is no preview of the files the next run would hit, and no test mode as in the file firewall. Try new retention periods first on a dedicated tag with only a few files.
• One retention period per tag. Retention periods cannot be combined or graded by user, group, or folder; the only lever is where the tag is set.
• No immediate effect. A newly added retention period applies at the next run of the worker job at the earliest, so within 24 hours with the tagbased engine.
• No control via occ. Retention periods are managed only in the admin settings; the app has no occ commands of its own.

Delete old files by age #

The oco_expire app (called "Auto Expire" in the Market app) works without tags. Once a day it moves every file in the accounts' own folders to the trash bin if the file was uploaded and last changed more than the configured number of days ago. Recently edited and freshly uploaded files therefore stay, even if a sync client sets an old modification time.

The settings are under Settings → Admin → Workflows & Tags, section Delete old files automatically:

Folders, external storages, and received shares are not touched. If an excluded group cannot be found, for example because it was deleted or the directory server does not answer, the run deletes nothing. If the trash bin is disabled, the app deletes files permanently; the settings page then shows a warning.

Before you switch it on, and before you reduce the number of days, a dry run shows what the next run would move to the trash bin:

sudo -u www-data php8.4 occ oco_expire:run --dry-run
sudo -u www-data php8.4 occ oco_expire:run --dry-run --user alice

Without --dry-run, occ oco_expire:run runs immediately, provided the option is switched on.

Check the state #

# Rules, log level, and hit counters of the firewall
sudo -u www-data php8.4 occ firewall:list

# Are the retention jobs running? The "Job" column shows the class names
sudo -u www-data php8.4 occ background:queue:status

# Run a specific job right away for checking
sudo -u www-data php8.4 occ background:queue:execute <job-id> --force

background:queue:execute is a maintenance command and asks for confirmation before it runs (--accept-warning skips the question); the regular schedule is not affected.

Troubleshooting #