Skip to content

elasticsearch

ElasticTransformer (BaseTransformer)

Transformer that transforms v0.10.1 grammar parse trees into queries.

Uses elasticsearch_dsl and will produce a Q instance.

__default__(self, tree, children, *args, **kwargs) special

Default behavior for rules that only replace one symbol with another

Source code in optimade/filtertransformers/elasticsearch.py 
def __default__(self, tree, children, *args, **kwargs):
    """ Default behavior for rules that only replace one symbol with another """
    return children[0]

__init__(self, quantities) special

Parameters:

Name Type Description Default
quantities List[optimade.filtertransformers.elasticsearch.Quantity]

A list of :class:Quantitys that describe how optimade (and other) quantities are mapped to the elasticsearch index.

 required
Source code in optimade/filtertransformers/elasticsearch.py 
def __init__(self, quantities: List[Quantity]):
    """
    Arguments:
        quantities: A list of :class:`Quantity`s that describe how optimade (and other)
            quantities are mapped to the elasticsearch index.
    """
    self.index_mapping = {quantity.name: quantity for quantity in quantities}
    super().__init__()

expression(self, args)

expression: expression_clause ( OR expression_clause )

Source code in optimade/filtertransformers/elasticsearch.py 
def expression(self, args):
    # expression: expression_clause ( _OR expression_clause )*
    result = args[0]
    for arg in args[1:]:
        result |= arg
    return result

expression_clause(self, args)

expression_clause: expression_phrase ( AND expression_phrase )*

Source code in optimade/filtertransformers/elasticsearch.py 
def expression_clause(self, args):
    # expression_clause: expression_phrase ( _AND expression_phrase )*
    result = args[0]
    for arg in args[1:]:
        result &= arg
    return result

expression_phrase(self, args)

expression_phrase: [ NOT ] ( comparison | "(" expression ")" )

Source code in optimade/filtertransformers/elasticsearch.py 
def expression_phrase(self, args):
    # expression_phrase: [ NOT ] ( operator | "(" expression ")" )
    if args[0] == "NOT":
        return ~args[1]
    return args[0]

filter(self, args)

filter: expression*

Source code in optimade/filtertransformers/elasticsearch.py 
def filter(self, args):
    # filter: expression*
    if len(args) == 1:
        return args[0]
    return Q("bool", **{"must": args})

fuzzy_string_op_rhs(self, args)

fuzzy_string_op_rhs: CONTAINS value | STARTS [ WITH ] value | ENDS [ WITH ] value

Source code in optimade/filtertransformers/elasticsearch.py 
def fuzzy_string_op_rhs(self, args):
    op = args[0]
    value = args[-1]
    if op == "CONTAINS":
        wildcard = "*%s*" % value
    if op == "STARTS":
        wildcard = "%s*" % value
    if op == "ENDS":
        wildcard = "*%s" % value

    return lambda quantity: Q("wildcard", **{self._field(quantity): wildcard})

length_op_rhs(self, args)

length_op_rhs: LENGTH [ OPERATOR ] value

Source code in optimade/filtertransformers/elasticsearch.py 
def length_op_rhs(self, args):
    # length_op_rhs: LENGTH [ OPERATOR ] signed_int
    value = args[-1]
    if len(args) == 3:
        op = args[1]
    else:
        op = "="

    def query(quantity):
        if quantity.length_quantity is None:
            raise Exception("LENGTH is not supported for %s" % quantity.name)
        quantity = quantity.length_quantity
        return self._query_op(quantity, op, value)

    return query

property(self, args)

property: IDENTIFIER ( "." IDENTIFIER )*

Source code in optimade/filtertransformers/elasticsearch.py 
def property(self, args):
    # property: IDENTIFIER ( "." IDENTIFIER )*
    quantity_name = args[0]

    if quantity_name not in self.index_mapping:
        raise Exception("%s is not a searchable quantity" % quantity_name)

    quantity = self.index_mapping.get(quantity_name, None)
    if quantity is None:
        quantity = Quantity(name=quantity_name)

    return quantity

property_zip_addon(self, args)

property_zip_addon: ":" property (":" property)*

Source code in optimade/filtertransformers/elasticsearch.py 
def property_zip_addon(self, args):
    return args

set_op_rhs(self, args)

set_op_rhs: HAS ( [ OPERATOR ] value | ALL value_list | ANY value_list | ONLY value_list )

Source code in optimade/filtertransformers/elasticsearch.py 
def set_op_rhs(self, args):
    # set_op_rhs: HAS ( [ OPERATOR ] value | ALL value_list | ... )
    values = args[-1]
    if not isinstance(values, list):
        if len(args) == 3:
            op = args[1]
        else:
            op = "="
        values = [(op, values)]

    if len(args) == 3:
        op = "HAS " + args[1]
    else:
        op = "HAS"

    return lambda quantity: self._has_query_op(
        [quantity], op, [[value] for value in values]
    )

set_zip_op_rhs(self, args)

set_zip_op_rhs: property_zip_addon HAS ( value_zip | ONLY value_zip_list | ALL value_zip_list | ANY value_zip_list )

Source code in optimade/filtertransformers/elasticsearch.py 
def set_zip_op_rhs(self, args):
    # set_zip_op_rhs: property_zip_addon HAS ( value_zip | ONLY value_zip_list | ALL value_zip_list | ANY value_zip_list )
    add_on = args[0]
    values = args[-1]
    if len(args) == 4:
        op = "HAS " + args[2]
    else:
        op = "HAS"
        values = [values]

    return lambda quantity: self._has_query_op([quantity] + add_on, op, values)

value_list(self, args)

value_list: [ OPERATOR ] value ( "," [ OPERATOR ] value )*

Source code in optimade/filtertransformers/elasticsearch.py 
def value_list(self, args):
    result = []
    op = "="
    for arg in args:
        if arg in ["<", "<=", ">", ">=", "!=", "="]:
            op = arg
        else:
            result.append(
                (
                    op,
                    arg,
                )
            )
            op = "="
    return result

value_zip(self, args)

value_zip: [ OPERATOR ] value ":" [ OPERATOR ] value (":" [ OPERATOR ] value)*

Source code in optimade/filtertransformers/elasticsearch.py 
def value_zip(self, args):
    return self.value_list(args)

value_zip_list(self, args)

value_zip_list: value_zip ( "," value_zip )*

Source code in optimade/filtertransformers/elasticsearch.py 
def value_zip_list(self, args):
    return args

Quantity

Class to provide information about available quantities to the transformer.

The elasticsearch transformer will use Quantitys to (a) do some semantic checks, (b) map quantities to the underlying elastic index.

Attributes:

Name Type Description
name

The name of the quantity as used in the filter expressions.
es_field

The name of the field for this quanity in elastic search, will be name by default.
elastic_mapping_type

A decendent of an elasticsearch_dsl Field that denotes which mapping was used in the elastic search index.
length_quantity

Elasticsearch does not support length of arrays, but we can map fields with array to other fields with ints about the array length. The LENGTH operator will only be supported for quantities with this attribute.
has_only_quantity

Elasticsearch does not support exclusive search on arrays, like a list of chemical elements. But, we can order all elements by atomic number and use a keyword field with all elements to perform this search. This only works for elements (i.e. labels in CHEMICAL_SYMBOLS) and quantities with this attribute.
nested_quantity

To support optimade's 'zipped tuple' feature (e.g. 'elements:elements_ratios HAS "H":>0.33), we use elasticsearch nested objects and nested queries. This quantity will provide the field for the nested object that contains the quantity (and others). The zipped tuples will only work for quantities that share the same nested object quantity.