Referring to components of Records - normalize.selector

There are two main purposes of normalize.selector:

• to abstract expressions to individual fields within objects, so that the result from a normalize.diff.diff() operation can specify the field where the difference(s) were discovered.
• to aggregate collections of these expressions, so that sets of attributes can be manipulated

The feature set builds on these basic purposes, starting with retrieval (see normalize.selector.FieldSelector.get()), assignment (see normalize.selector.FieldSelector.put()), assignment with auto-vivification of intermediate records and collection items (normalize.selector.FieldSelector.post()), and finally deletion (normalize.selector.FieldSelector.delete()).

Multple FieldSelector objects can be combined, to make a MultiFieldSelector. It also supports normalize.selector.MultiFieldSelector.get() which returns a “filtered” object, normalize.selector.MultiFieldSelector.patch() which can be used to selectively assign values from one object to another, and normalize.selector.MultiFieldSelector.delete().

The MultiFieldSelector can be used to restrict the action of visitor functions (such as normalize.diff.diff() and normalize.visitor.VisitorPattern) to compare or visit only a selection of fields in a data structure.

Class reference

class normalize.selector.FieldSelector(expr=None)

A way to refer to fields/record/properties within a data structure.

This is modeled as a list of either attribute names or collection indices, where:

• A string specifies an attribute of an object (or a dictionary key)
• An integer specifies an index into a collection.
• ‘None’ specifies the full collection.

__init__(expr=None)

Initializer for FieldSelector instances.

args:

expr=FieldSelector|<iterable>

Starting expression for the selector; can copy an existing FieldSelector or instantiate one from an iterable list of attribute names/indices

add_property(prop)

Extends the selector, adding a new attribute property lookup at the end, specified by name.

add_index(index)

Extends the selector, adding a new indexed collection lookup at the end.

add_full_collection()

Extends the selector, making it refer to all items in the collection at this point.

extend(other)

Extends this field selector with another FieldSelector, combining them to one longer field selector.

__getnewargs__()

The pickle protocol is supported on this type.

get(record)

Evaluate the FieldSelector’s path to get a specific attribute (or collection of attributes, if there is a None in the selector expression) from the passed record.

If there is a problem, this method will typically raise FieldSelectorException.

If the FieldSelector contains None, then exceptions from iterating over all of the sub-fields are suppressed, so long as one of the items succeed. The return value will have None for the values which did not have the attribute. If all the items fail, then one of the exceptions will be bubbled up to the caller.

record.foo = "bar"
field_selector = FieldSelector(["foo"])
print field_selector.get(record)  # "bar"

put(record, value)

Sets the field referenced by this field selector for the given Record with the given value.

field_selector.put(record, "baz")
print record.foo  # "baz"

post(record, value)

auto-vivifying version of ‘put’; if properties are not found along the path, attempts are made to set them to empty values.

Returns the number of values set; may be 0 if there is ‘None’ in the selector and there were no existing items in that collection.

field_selector = FieldSelector(["stuff", 0, "name"])
obj = SomeObject()
field_selector.post(obj, "Bob")
print obj.stuff[0].name  # "Bob"

delete(record)

Like ‘put’, but deletes the item from the specified location.

If there is a problem, this method will typically raise FieldSelectorException.

This function has the same behavior with respect to collections as get(): it will only raise an exception if all of the items in the set did not have the property to delete.

record.foo = "bar"
field_selector = FieldSelector(["foo"])
field_selector.delete(record)
print field_selector.foo  # AttributeError

__eq__(other)

Implemented; field selectors must have identical paths to compare equal.

__ne__(other)

functools.total_ordering() currently doesn’t take care of __ne__(). Don’t call other comparison methods directly to avoid infinite recursion.

__lt__(other)

Ordering field selectors makes sure that all integer-indexed selectors are incrementing. This is mainly needed for FieldSelector.post(), which will only auto-extend collections items at the end.

__str__()

Returns a compact representation of the field selector, shown as a python expression.

__repr__()

Returns a evaluable representation of a field selector.

__add__(other)

Creates a new FieldSelector, with the two attribute/index expression lists concatenated. Like extend() but creates a new FieldSelector.

fs = FieldSelector(["foo"])
bar = FieldSelector(["bar"])

print fs + bar  # <FieldSelector: .foo.bar>
print fs + [0]  # <FieldSelector: .foo[0]>

__len__()

Returns the number of elements in the field selector expression.

__getitem__(key)

Indexing can be used to return a particular item from the selector expression, and slicing can be used to make a new FieldSelector which has a shorter expression. For instance, to trim the last:

trimmed_fs = field_selector[:-1]

startswith(key_or_fs)

Can be used to assert how the field selector begins.

args:

key_or_fs=FieldSelector|<attribute-or-index>

If the argument is another FieldSelector (or a tuple/list), it checks that the invocant’s first selector expression components match the components in the selector passed.

If the argument is a valid index/key or attribute name, it will check that the first member in the expression is the same as that passed.

path

This property returns something that looks a bit like a python representation of the implied expression.

foo = FieldSelector(["foo", 2, "b ar", None, "baz"])
print foo.path  # foo[2]['b ar'][*].baz

classmethod from_path(path)

Alternate constructor. Constructs a FieldSelector from the abbreviated text representation (.path attribute)

class normalize.selector.MultiFieldSelector(*others)

Version of a FieldSelector which stores multiple FieldSelectors combined into a single tree structure.

__init__(*others)

Returns a MultiFieldSelector based on combining the passed-in FieldSelector and MultiFieldSelector objects.

args:

*others=FieldSelector|iterable

Each argument is interpreted as either a FieldSelector, or a FieldSelector constructor.

__str__()

Stringification of a MultiFieldSelector shows just the keys in the top level, and an indication of whether a sub-key is filtered.

>>> mfs = MultiFieldSelector(["a", "b"], ["a", "d"], ["c"])
>>> str(mfs)
'<MultiFieldSelector: a.b|a.d|c>
>>>

path

The path attribute returns a stringified, concise representation of the MultiFieldSelector. It can be reversed by the from_path constructor.

__iter__()

Generator for all FieldSelectors this MultiFieldSelector implicitly contains.

>>> mfs = MultiFieldSelector(["a", "b"], ["a", "d"], ["c"])
>>> for x in mfs:
...     print x
...
<FieldSelector: .a.b>
<FieldSelector: .a.d>
<FieldSelector: .c>
>>>

__getitem__(index)

Returns the sub-MultiFieldSelector that applies to the specified field/key/index. If the MultiFieldSelector does not match any fields at this path, it returns None. If it matches all fields at the path, then it returns a ‘complete’ MultiFieldSelector (ie, a field selector which matches everything). If it matches a subset of fields at that location, it will return a MultiFieldSelector which contains those fields.

>>> mfs = MultiFieldSelector(["a", "b"], ["a", "d"], ["c"])
>>> mfs["a"]
MultiFieldSelector(['b'], ['d'])
>>> mfs[("a", "c")]
None

__contains__(index)

Checks to see whether the given item matches the MultiFieldSelector.

For item index (string, number, None) lookups, the result is True if the result of applying the MultiFieldSelector (with .get) would return an item in that location (if it existed in the input).

For a FieldSelector (or sequence) lookups, the result is True if the result of applying the MultiFieldSelector and then the FieldSelector on an input is the same as applying the FieldSelector on the input. ie, that the MultiFieldSelector is a superset of the FieldSelector.

Unlike plain collections, use of in does not mean that subscripting will return KeyError. When used with a FieldSelector, it returns False when __getitem__ with the index would not return a complete FieldSelector, whereas the __getitem__ call would return a partial (non-complete) MultiFieldSelector.

>>> mfs = MultiFieldSelector(["a", "b"], ["a", "d"], ["c"])
>>> "a" in mfs
True
>>> "b" in mfs
False
>>> FieldSelector(["a"]) in mfs
False
>>> FieldSelector(["a", "d"]) in mfs
True
>>> FieldSelector(["a", "e"]) in mfs
False
>>>

__repr__()

Implemented as per SPECIALMETHODS recommendation to return a full python source to reconstruct:

>>> mfs = MultiFieldSelector(["a", "b"], ["a", "d"], ["c"])
>>> mfs
MultiFieldSelector(['a', 'b'], ['a', 'd'], ['c'])
>>>

get(obj)

Creates a copy of the passed object which only contains the parts which are pointed to by one of the FieldSelectors that were used to construct the MultiFieldSelector. Can be used to produce ‘filtered’ versions of objects.

delete(obj, force=False)

Deletes all of the fields at the specified locations.

args:

obj=OBJECT

the object to remove the fields from

force=BOOL

if True, missing attributes do not raise errors. Otherwise, the first failure raises an exception without making any changes to obj.

patch(target, source, copy=False)

Copies fields from obj to target. If a matched field does not exist in obj, it will be deleted from target, otherwise it will be assigned (or copied).

args:

target=OBJECT

the object to set the fields in

source=OBJECT

the object to lift the fields from

copy=BOOL|FUNCTION

deep copy the values set, using copy.deepcopy (or the passed function). False by default.

classmethod from_path(mfs_path)

Alternate constructor. Constructs a MultiFieldSelector from the abbreviated text representation (.path attribute)