The rule object

The rule object returns an individual rule for the robots.txt file, which tells crawlers which pages can, or can't, be accessed. It consists of a directive, which can be either Allow or Disallow, and a value of the associated URL path.

For example:

The rule object has the following attributes: