Lenient Metadata¶

This section discusses lenient metadata; what it is, what it means, and how you can perform lenient rather than strict operations with your metadata.

Introduction¶

As discussed in Metadata, a rich, common metadata API is available within Iris that supports metadata equality, difference, combination, and also conversion.

The common metadata API is implemented through the metadata property on each of the Iris CF Conventions class containers (Table 2), and provides a common gateway for users to easily manage and manipulate their metadata in a consistent and unified way.

This is primarily all thanks to the metadata classes (Table 2) that support the necessary state and behaviour required by the common metadata API. Namely, it is the equal (__eq__), difference and combine methods that provide this rich metadata behaviour, all of which are explored more fully in Metadata.

Strict Behaviour¶

The feature that is common between the equal, difference and combine metadata class methods, is that they all perform strict metadata member comparisons by default.

The strict behaviour implemented by these methods can be summarised as follows, where X and Y are any objects that are non-identical,

Table 3 - Strict equality¶
Left	Right	`equal`
`X`	`Y`	`False`
`Y`	`X`	`False`
`X`	`X`	`True`
`X`	`None`	`False`
`None`	`X`	`False`

Table 4 - Strict difference¶
Left	Right	`difference`
`X`	`Y`	(`X`, `Y`)
`Y`	`X`	(`Y`, `X`)
`X`	`X`	`None`
`X`	`None`	(`X`, `None`)
`None`	`X`	(`None`, `X`)

Table 5 - Strict combination¶
Left	Right	`combine`
`X`	`Y`	`None`
`Y`	`X`	`None`
`X`	`X`	`X`
`X`	`None`	`None`
`None`	`X`	`None`

This type of strict behaviour does offer obvious benefit and value. However, it can be unnecessarily restrictive. For example, consider the metadata of the following latitude coordinate,

>>> latitude.metadata
DimCoordMetadata(standard_name='latitude', long_name=None, var_name='latitude', units=Unit('degrees'), attributes={}, coord_system=GeogCS(6371229.0), climatological=False, circular=False)

Now, let’s create a doctored version of this metadata with a different var_name,

>>> metadata = latitude.metadata._replace(var_name=None)
>>> metadata
DimCoordMetadata(standard_name='latitude', long_name=None, var_name=None, units=Unit('degrees'), attributes={}, coord_system=GeogCS(6371229.0), climatological=False, circular=False)

Clearly, these metadata are different,

>>> metadata != latitude.metadata
True
>>> metadata.difference(latitude.metadata)
DimCoordMetadata(standard_name=None, long_name=None, var_name=(None, 'latitude'), units=None, attributes=None, coord_system=None, climatological=None, circular=None)

And yet, they both have the same name, which some may find slightly confusing (see name() for clarification)

>>> metadata.name()
'latitude'
>>> latitude.name()
'latitude'

Resolving this metadata inequality can only be overcome by ensuring that each metadata member precisely matches.

If your workflow demands such metadata rigour, then the default strict behaviour of the common metadata API will satisfy your needs. Typically though, such strictness is not necessary, and as of Iris 3.0.0 an alternative more practical behaviour is available.

Lenient Behaviour¶

Lenient metadata aims to offer a practical, common sense alternative to the strict rigour of the default Iris metadata behaviour. It is intended to be complementary, and suitable for those users with a more relaxed requirement regarding their metadata.

The lenient behaviour that is implemented as an alternative to the strict equality, strict difference, and strict combination can be summarised as follows,

Table 6 - Lenient equality¶
Left	Right	`equal`
`X`	`Y`	`False`
`Y`	`X`	`False`
`X`	`X`	`True`
`X`	`None`	`True`
`None`	`X`	`True`

Table 7 - Lenient difference¶
Left	Right	`difference`
`X`	`Y`	(`X`, `Y`)
`Y`	`X`	(`Y`, `X`)
`X`	`X`	`None`
`X`	`None`	`None`
`None`	`X`	`None`

Table 8 - Lenient combination¶
Left	Right	`combine`
`X`	`Y`	`None`
`Y`	`X`	`None`
`X`	`X`	`X`
`X`	`None`	`X`
`None`	`X`	`X`

Lenient behaviour is enabled for the equal, difference, and combine metadata class methods via the lenient keyword argument, which is False by default. Let’s first explore some examples of lenient equality, difference and combination, before going on to clarify which metadata members adopt lenient behaviour for each of the metadata classes.

Lenient Equality¶

Lenient equality is enabled using the lenient keyword argument, therefore we are forced to use the equal method rather than the == operator (__eq__). Otherwise, the equal method and == operator are both functionally equivalent.

For example, consider the previous strict example, where two separate latitude coordinates are compared, each with different var_name members,

>>> metadata.equal(latitude.metadata, lenient=True)
True

Unlike strict comparison, lenient comparison is a little more forgiving. In this case, leniently comparing something with nothing (None) will always be True; it’s the graceful compromise to the strict alternative.

So let’s take the opportunity to reinforce this a little further before moving on, by leniently comparing different attributes dictionaries; a constant source of strict contention.

Firstly, populate the metadata of our latitude coordinate appropriately,

>>> attributes = {"grinning face": "😀", "neutral face": "😐"}
>>> latitude.attributes = attributes
>>> latitude.metadata  
DimCoordMetadata(standard_name='latitude', long_name=None, var_name='latitude', units=Unit('degrees'), attributes={'grinning face': '😀', 'neutral face': '😐'}, coord_system=GeogCS(6371229.0), climatological=False, circular=False)

Then create another DimCoordMetadata with a different attributes dict, namely,

the grinning face key is missing,
the neutral face key has the same value, and
the upside-down face key is new

>>> attributes = {"neutral face": "😐", "upside-down face": "🙃"}
>>> metadata = latitude.metadata._replace(attributes=attributes)
>>> metadata  
DimCoordMetadata(standard_name='latitude', long_name=None, var_name='latitude', units=Unit('degrees'), attributes={'neutral face': '😐', 'upside-down face': '🙃'}, coord_system=GeogCS(6371229.0), climatological=False, circular=False)

Now, compare our metadata,

>>> metadata.equal(latitude.metadata)
False
>>> metadata.equal(latitude.metadata, lenient=True)
True

Again, lenient equality (Table 6) offers a more forgiving and practical alternative to strict behaviour.

Lenient Difference¶

Similar to Lenient Equality, the lenient difference method (Table 7) considers there to be no difference between comparing something with nothing (None). This working assumption is not naively applied to all metadata members, but rather a more pragmatic approach is adopted, as discussed later in Lenient Members.

Again, lenient behaviour for the difference metadata class method is enabled by the lenient keyword argument. For example, consider again the previous strict example involving our latitude coordinate,

>>> metadata.difference(latitude.metadata)
DimCoordMetadata(standard_name=None, long_name=None, var_name=(None, 'latitude'), units=None, attributes=None, coord_system=None, climatological=None, circular=None)
>>> metadata.difference(latitude.metadata, lenient=True) is None
True

And revisiting our slightly altered attributes member comparison example, brings home the benefits of the lenient difference behaviour. So, given our latitude coordinate with its populated attributes dictionary,

>>> latitude.attributes  
{'grinning face': '😀', 'neutral face': '😐'}

We create another DimCoordMetadata with a dissimilar attributes member, namely,

the grinning face key is missing,
the neutral face key has a different value, and
the upside-down face key is new

>>> attributes = {"neutral face": "😜", "upside-down face": "🙃"}
>>> metadata = latitude.metadata._replace(attributes=attributes)
>>> metadata  
DimCoordMetadata(standard_name='latitude', long_name=None, var_name='latitude', units=Unit('degrees'), attributes={'neutral face': '😜', 'upside-down face': '🙃'}, coord_system=GeogCS(6371229.0), climatological=False, circular=False)

Now comparing the strict and lenient behaviour for the difference method, highlights the change in how such dissimilar metadata is treated gracefully,

>>> metadata.difference(latitude.metadata).attributes  
{'upside-down face': '🙃', 'neutral face': '😜'}, {'neutral face': '😐', 'grinning face': '😀'}
>>> metadata.difference(latitude.metadata, lenient=True).attributes  
{'neutral face': '😜'}, {'neutral face': '😐'}

Lenient Combination¶

The behaviour of the lenient combine metadata class method is outlined in Table 8, and as with Lenient Equality and Lenient Difference is enabled through the lenient keyword argument.

The difference in behaviour between lenient and strict combination is centred around the lenient handling of combining something with nothing (None) to return something. Whereas strict combination will only return a result from combining identical objects.

Again, this is best demonstrated through a simple example of attempting to combine partially overlapping attributes member dictionaries. For example, given the following attributes dictionary of our favoured latitude coordinate,

>>> latitude.attributes  
{'grinning face': '😀', 'neutral face': '😐'}

We create another DimCoordMetadata with overlapping keys and values, namely,

the grinning face key is missing,
the neutral face key has the same value, and
the upside-down face key is new

>>> attributes = {"neutral face": "😐", "upside-down face": "🙃"}
>>> metadata = latitude.metadata._replace(attributes=attributes)
>>> metadata  
DimCoordMetadata(standard_name='latitude', long_name=None, var_name='latitude', units=Unit('degrees'), attributes={'neutral face': '😐', 'upside-down face': '🙃'}, coord_system=GeogCS(6371229.0), climatological=False, circular=False)

Comparing the strict and lenient behaviour of combine side-by-side highlights the difference in behaviour, and the advantages of lenient combination for more inclusive, richer metadata,

>>> metadata.combine(latitude.metadata).attributes
{'neutral face': '😐'}
>>> metadata.combine(latitude.metadata, lenient=True).attributes  
{'neutral face': '😐', 'upside-down face': '🙃', 'grinning face': '😀'}

Lenient Members¶

Lenient Behaviour is not applied regardlessly across all metadata members participating in a lenient equal, difference or combine operation. Rather, a more pragmatic application is employed based on the CF Conventions definition of the member, and whether being lenient would result in erroneous behaviour or interpretation.

Table 9 - Lenient member participation¶
Metadata Class	Member	Behaviour
All metadata classes†	`standard_name`	`lenient`‡
All metadata classes†	`long_name`	`lenient`‡
All metadata classes†	`var_name`	`lenient`‡
All metadata classes†	`units`	`strict`
All metadata classes†	`attributes`	`lenient`
`CellMeasureMetadata`	`measure`	`strict`
`CoordMetadata`, `DimCoordMetadata`	`coord_system`	`strict`
`CoordMetadata`, `DimCoordMetadata`	`climatological`	`strict`
`CubeMetadata`	`cell_methods`	`strict`
`DimCoordMetadata`	`circular`	`strict` §

Key
† - Applies to all metadata classes including AncillaryVariableMetadata, which has no other specialised members
‡ - See Special Lenient Name Behaviour for standard_name, long_name, and var_name
§ - The circular is ignored for operations between CoordMetadata and DimCoordMetadata

In summary, only standard_name, long_name, var_name and the attributes members are treated leniently. All other members are considered to represent fundamental metadata that cannot, by their nature, be consider equivalent to metadata that is missing or None. For example, a Cube with units of ms-1 cannot be considered equivalent to another Cube with units of unknown; this would be a false and dangerous scientific assumption to make.

Similar arguments can be made for the measure, coord_system, climatological, cell_methods, and circular members, all of which are treated with strict behaviour, regardlessly.

Special Lenient Name Behaviour¶

The standard_name, long_name and var_name have a closer association with each other compared to all other metadata members, as they all underpin the functionality provided by the name() method. It is imperative that the name() derived from metadata remains constant for strict and lenient equality alike.

As such, these metadata members have an additional layer of behaviour enforced during Lenient Equality in order to ensure that the identity or name of metadata does not change due to a side-effect of lenient comparison.

For example, if simple lenient equality behaviour was applied to the standard_name, long_name and var_name, the following would be considered not equal,

Member	Left	Right
`standard_name`	`None`	`latitude`
`long_name`	`latitude`	`None`
`var_name`	`lat`	`latitude`

Both the Left and Right metadata would have the same name() by definition i.e., latitude. However, lenient equality would fail due to the difference in var_name.

To account for this, lenient equality is performed by two simple consecutive steps:

ensure that the result returned by the name() method is the same for the metadata being compared, then
only perform lenient equality between the standard_name and long_name i.e., the var_name member is not compared explicitly, as its value may have been accounted for through name() equality

Metadata

Lenient Cube Maths