Chapter 8 Rules in text files

This Chapter is about importing and exporting rules from and to file, both in free-form text and in YAML. We also discuss some more advanced features like how to have one rule file include another file.

8.1 Reading rules from file

It is a very good idea to store and maintain rule sets outside of your R script. Validate supports two file formats: simple text files and yaml files. Here we only discuss simple text files, yaml files are treated in 8.2.

To try this, copy the following rules into a new text file and store it in a file called myrules.R, in the current working directory of your R session.

# basic range checks
speed >= 0
dist  >= 0

# ratio check
speed / dist <= 1.5

Note that you are allowed to annotate the rules as you would with regular R code. Reading these rules can be done as follows.

rules <- validator(.file="myrules.R")

8.2 Metadata in text files: `YAML`

YAML is a data format that aims to be easy to learn and human-readable. The name ‘YAML’ is a recursive acronym that stands for

YAML Ain’t Markup Language.

Validate can read and write rule sets from and to YAML files. For example, paste the following code into a file called myrules.yaml.

rules:
- expr: speed >= 0
  name: 'speed'
  label: 'speed positivity'
  description: |
    speed can not be negative
  created: 2020-11-02 11:15:11
  meta:
    language: validate 0.9.3.36
    severity: error
- expr: dist >= 0
  name: 'dist'
  label: 'distance positivity'
  description: |
    distance cannot be negative.
  created: 2020-11-02 11:15:11
  meta:
    language: validate 0.9.3.36
    severity: error
- expr: speed/dist <= 1.5
  name: 'ratio'
  label: 'ratio limit'
  description: | 
    The speed to distance ratio can
    not exceed 1.5.
  created: 2020-11-02 11:15:11
  meta:
    language: validate 0.9.3.36
    severity: error

We can read this file using validator(.file=) as before.

rules <- validator(.file="myrules.yaml")
rules

## Object of class 'validator' with 3 elements:
##  speed [speed positivity]  : speed >= 0
##  dist [distance positivity]: dist >= 0
##  ratio [ratio limit]       : speed/dist <= 1.5

Observe that the labels are printed between brackets. There are a few things to note about these YAML files.

rules: starts a list of rules.
Each new rule starts with a dash (-)
Each element of a rule is denoted name: <content>. The only obligated element is expr: the rule expression.
Spaces matter. Each element of a rule must be preceded by a newline and two spaces. Subelements (as in meta) are indented again.

A full tutorial on YAML can be found at W3Cschools.io.

To export a rule set to yaml, use the export_yaml() function.

rules1 <- rules[c(1,3)]
export_yaml(rules1, file="myrules2.yaml")

8.3 Setting options

Both free-form and YAML files can optionally start with a header section where options or file inclusions can be set. The header section is enclosed by lines that contain three dashes (---) at the beginning of the line.

For example, in the following rule file we make sure that errors are not caught but raised to run-time level, and we set the tolerance for checking linear equalities and inequalities to zero.

---
options:
  raise: errors
  lin.eq.eps: 0
  lin.ineq.eps: 0
---

turnover >= 0

staff >= 0

total.rev - profit == total.costs

The options you set here will be part of the validator object, that is created once you read in the file. The options are valid for every confrontation you use this validator for, unless they are overwritten during the call to confront().

The header section is interpreted as a block of YAML, so options and file inclusions must be specified in that format.

8.4 Including other rule files

In validate, rule files can include each other recursively. So file A can include file B, which may include file C. This is useful for example in surveys where the first part of the questionnaire goes to all respondents, and for the second part, the contents of the questionnaire (and hence its variables) depend on the respondent type. One could create files with specific rules for the second part: one for each respondent group, and have each specific rule file include the general rules that must hold for every respondent. It can also be useful when different persons are responsible for different rule sets.

File inclusion can be set through the include option in the YAML header.

---
include:
  - petes_rules.yaml
  - nancys_rules.yaml
options:
  raise: errors
---
# start rule definitions here

8.5 Exporting validator objects

There are three ways to do that. You can either write to a yaml file immediately as follows

v <- validator(height>0, weight> 0)
export_yaml(v,file="my_rules.yaml")

or you can get the yaml text string using as_yaml

cat(as_yaml(v))

Finally, you can convert a rule set to data frame and then export it to a database.

df <- as.data.frame(v)