/
XPath-like Query Language: Structure, Types, and Efficiency in Metadata Searching

XPath-like Query Language: Structure, Types, and Efficiency in Metadata Searching

EIDR provides a powerful query facility that serves as the primary mechanism for searching the EIDR content record database, beyond simple ID resolution. This query language leverages an XPath-like syntax to allow users to construct detailed search expressions based on metadata fields.

Query Syntax and Structure:

  • EIDR queries use a simplified form of XPath notation to specify metadata fields.

  • To identify a data element, you start with a root node and traverse through the nested XML structure, separating each element name with a forward slash (/).

    ◦ /FullMetadata: Used for querying across the object's inherited metadata (content records).

    ◦ /ProvenanceMetadata: Used for querying across the object's creation/modification history.

    ◦ /Service and /Party: Used for querying Video Service and Party records, respectively.

  • Attributes of an XML element are separated from the element name using an at-symbol (@) instead of a slash. For example, the domain attribute of an AlternateID is expressed as /FullMetadata/BaseObjectData/AlternateID@domain.

  • Parentheses are often used to improve legibility, especially with complex expressions.

Types of Queries:

  1. General Queries: These return all content records that match the specified criteria.

  1. Rooted Queries: These return only content records that meet the query criteria and are descended from a specified EIDR ID (e.g., finding all episodes within a particular series).

Query Components and Operators: EIDR's query language supports various components and operators to build search expressions:

  • Existence: EXISTS checks for the presence of a field (e.g., /FullMetadata/ExtraObjectMetadata/ManifestationInfo/Digital/Track/Subtitle EXISTS).

  • Text Matching:

    ◦ Token Match: <field> <string1> <string2>* is true if the field contains one or more of the strings, in any order (equivalent to OR conditions).

    ◦ Exact String Match: <field> "<string>" requires the exact token sequence of the string to appear in the field.

    ◦ Text queries are generally case-insensitive and involve tokenization (normalization, punctuation handling). For Description fields, stop words are filtered and words are stemmed.

    ◦ ASCII Operator: ASCII (<expression>) changes how Latin alphabet-based text strings are compared, evaluating characters with and without diacritical marks as identical (e.g., "ü" is equivalent to "u").

  • Exact Value: IS and ISNOT are used for fields containing DOIs, controlled vocabulary, or certain text fields. ISNOT returns TRUE for non-existent fields.

  • Order/Comparison: Operators like =, <>, <, <=, >, >= can be used for fields containing integers, dates, or durations (e.g., /FullMetadata/BaseObjectData/ApproximateLength > PT20M00S).

  • Logical Expressions: Queries can be combined using AND, OR, and NOT.

Query Results and Efficiency:

  • Query results are returned in paged sets, allowing for efficient retrieval of large numbers of matches. Users can specify PageSize and PageNumber, and a ContinuationToken is used for subsequent requests.

  • For optimal efficiency, using a large PageSize is recommended, especially for non-interactive display of results.

  • By default, results are sorted alphabetically by the Resource Name (primary title).