What is zc.intid?

https://travis-ci.org/zopefoundation/zc.intid.svg?branch=master

zc.intid provides an API to create integer ids for any object. Objects can later be looked up by their id as well. This functionality is commonly used in situations where dealing with objects is undesirable, such as in search indices or any code that needs an easy hash of an object.

This is similar to the zope.intid package, but with the advantage of inducing fewer ZODB conflict errors, since object ids are not used as part of the stored data. The id for an object is stored in an attribute of the object itself, with the attribute name being configured by the construction of the id utility.

This does require that the object being registered “play nice” with this approach. At a minimum, the attributes used to store ids on objects should

  • persist with the rest of the object’s state, and
  • not be modified by the object.

Events generated on the assignment and removal of ids are generated by the register() and unregester() methods rather than by the callers of those methods.

Installation

zc.intid may be installed using pip:

pip install zc.intid

For information on configuring zc.intid, see Configuring.

Reference

Interfaces

Interfaces for the unique id utility.

Note that most of these interfaces present identical method signatures to those of their zope.intid counterparts. This includes everything that comprises the IIntIds interface.

Note that the contracts for these APIs differs, primarily in not requiring IKeyReference support (however, the provided lifecycle event subscribers in zc.intid.subscribers do require this support).

The IIntIdsSubclass and event interfaces are new.

exception zc.intid.interfaces.IntIdMismatchError[source]

Bases: zope.intid.interfaces.IntIdMissingError

Raised from getId if the id of an object doesn’t match what’s recorded in the utility.

exception zc.intid.interfaces.IntIdInUseError[source]

Bases: ValueError

Raised by the utility when register tries to reuse an intid.

interface zc.intid.interfaces.IIntIdsQuery[source]

Finding IDs by object and objects by ID.

getObject(uid)

Return an object by its unique id

Raises:zope.intid.interfaces.ObjectMissingError – if there is no object with that id.
getId(ob)

Get a unique id of an object.

Raises:
queryObject(uid, default=None)

Return an object by its unique id

Return the default if the uid isn’t registered

queryId(ob, default=None)

Get a unique id of an object.

Return the default if the object isn’t registered

__iter__()

Return an iteration on the ids

interface zc.intid.interfaces.IIntIdsSet[source]

Establishing and destroying the connection between an object and an ID.

register(ob)

Register an object and returns a unique id generated for it.

If the object is already registered, its id is returned anyway.

If not already registered, the registration is made and an IIdAddedEvent is generated.

unregister(ob)

Remove the object from the indexes.

If the ob is not previously registered, this has no effect.

An IIdRemovedEvent is triggered for successful unregistrations.

interface zc.intid.interfaces.IIntIdsManage[source]

Some methods used by the view.

__len__()

Return the number of objects indexed.

items()

Return a list of (id, object) pairs.

interface zc.intid.interfaces.IIntIds[source]

Extends: zc.intid.interfaces.IIntIdsSet, zc.intid.interfaces.IIntIdsQuery, zc.intid.interfaces.IIntIdsManage

A utility that assigns unique ids to objects.

Allows to query object by id and id by object.

interface zc.intid.interfaces.IIntIdsSubclass[source]

Additional interface that subclasses can usefully use.

family

BTree family used for this id utility.

This will be either BTree.family32 or BTree.family64.

This may not be modified, but may be used to create additional structures of the same integer family as the refs structure.

refs

BTree mapping from id to object.

Subclasses can use this to determine whether an id has already been assigned.

This should not be directly modified by subclasses.

generateId(ob)

Return a new iid that isn’t already used.

ob is the object the id is being generated for.

The default behavior is to generate arbitrary integers without reference to the objects they’re generated for.

This method may be overriden.

If this method returns an id that is already in use, register will raise an IntIdInUseError.

interface zc.intid.interfaces.IIdEvent[source]

Generic base interface for IntId-related events

object

The object related to this event

idmanager

The int id utility generating the event.

id

The id that is being assigned or unassigned.

interface zc.intid.interfaces.IIdRemovedEvent[source]

Extends: zc.intid.interfaces.IIdEvent

A unique id will be removed.

The event is published before the unique id is removed from the utility so that the indexing objects can unindex the object.

interface zc.intid.interfaces.IIdAddedEvent[source]

Extends: zc.intid.interfaces.IIdEvent

A unique id has been added.

The event gets sent when an object is registered in a unique id utility.

interface zc.intid.interfaces.ISubscriberEvent[source]

An event fired by the subscribers in relation to another event.

object

The object related to this event

original_event

The ObjectEvent related to this event

interface zc.intid.interfaces.IAfterIdAddedEvent[source]

Extends: zc.intid.interfaces.ISubscriberEvent

Fired after all utilities have registered unique ids.

This event is guaranteed to be the last event fired by the subscribers that register ids. It will be fired exactly once, no matter how many utilities registered ids.

This has a similar purpose and structure to zope.intid.interfaces.IIntIdAddedEvent.

idmap

The dictionary that holds an (utility -> id) mapping of created ids

interface zc.intid.interfaces.IBeforeIdRemovedEvent[source]

Extends: zc.intid.interfaces.ISubscriberEvent

Fired before any utility removes an object’s unique ID.

This event is guaranteed to be the first event fired by the subscriber that removes IDs. It will only be fired if at least one utility will remove an ID.

Implementation

Unique id utility.

This utility assigns unique integer ids to objects and allows lookups by object and by id.

This functionality can be used in cataloging.

class zc.intid.utility.IntIds(attribute, family=None)[source]

Bases: persistent.Persistent

This utility provides a two way mapping between objects and integer ids.

The objects are stored directly in the internal structures.

generateId(ob)[source]

Generate an id which is not yet taken.

This tries to allocate sequential ids so they fall into the same BTree bucket, and randomizes if it stumbles upon a used one.

Lifecycle Event Subscribers

A set of subscribers for the object zope.lifecycleevent events.

These subscribers take care of registering and unregistering objects with all available IIntIds utilities when IObjectAddedEvent and IObjectRemovedEvent events are fired, respectively.

These subscribers and events are modeled on those that come with zope.intid and are intended to be used (optionally) as drop-in replacements for them. This allows zc.intid to work in conjunction with things written for zope.intid, such as zope.catalog.

In particular, a few things are done just like zope.intid:

  1. We do ensure that the object can be adapted to IKeyReference before doing any processing (even though we don’t register that in the utility or otherwise use it.) In the common case of persistent objects, this will ensure that the object is in the database and has a jar and oid, common needs.
  2. We do broadcast the events from zope.intid.interfaces, even though the utility will broadcast its own events. Thus these subscribers generate at least three events for every lifecycle event.

Configuring

To configure, you need to include subscribers.zcml, while being careful about how zope.intid is configured:

<!-- configure.zcml -->
<!--
If we load zope.intid, we get subscribers for the Object events
that ensure all ILocation objects are registered/unregistered when
they are added/removed, plus another set of events when they
get/lose intids. This second set of events is meant to update
zope.catalog. A consequence of this is that ILocation objects must
be adaptable to KeyReferences when they are ObjectAdded (for
purposes of zope.intid, which we don't care about, but this also
ensures that they have ZODB Connections, which is good).

We cannot use these subscribers as-is due to the way the use IKeyReference
and try to register that. However, our subscribers *do* make sure that
the given objects can be adapted to IKeyReference because that's useful and
may be required by catalogs or other subscribers.
-->
<exclude package="zope.intid" file="subscribers.zcml" />
<include package="zope.intid" />

<!-- Make sure the default IKeyReference adapters are in place -->
<include package="zope.keyreference" />

<include package="zc.intid" />

<!--
Make zc.intid utilities compatible with zope.intid utilities.
-->
<include package="zc.intid" file="zope-intid.zcml" />

<!-- To hook them up to the Object events, we need to include the file -->
<include package="zc.intid" file="subscribers.zcml" />

KeyReferences and zope.intid

These subscribers do not register/unregister a IKeyReference with the intid utilities. Instead, it registers the actual object, and the events that are broadcast are broadcast holding the actual object.

IKeyReferenceces, especially KeyReferenceToPersistent, are used for a few reasons. First, they provide a stable, object-identity-based pointer to objects. To be identity based, this pointer is independent of the equality and hashing algorithms of the underlying object. Identity-based comparisons are necessary for the classic zope.intid utility implementation which uses a second OIBTree to maintain the backreferece from object to assigned intid (clearly you don’t want two non-identical objects which happen to compare equally now to get the same intid as that condition may change). Likewise, these references are all defined to be mutually comparable, no matter how they are implemented, a condition necessary for them to all work together in a OIBTree. Lastly, these references are meant to be comparable during ZODB conflict resolution (the original persistent objects probably won’t be), which, again, is a condition of the implementation using a OIBTree.

A consequence of avoiding these references is that generally persistent objects that are expected to have intids assigned should not be used as keys in an OxBTree or stored in an OOSet. Instead, all such data structures should use the integer variations (e.g., IISet), with the intid as the key.

Subscriber Functions

zc.intid.subscribers.addIntIdSubscriber(ob, event)[source]

Registers the object in all unique id utilities and fires an event for the catalogs. Notice that each utility will fire zc.intid.interfaces.IIdAddedEvent; this subscriber will then fire one single zope.intid.interfaces.IIntIdAddedEvent, followed by one single zc.intid.interfaces.IAfterIdAddedEvent; this gives a guaranteed order such that zope.catalog and other Zope event listeners will have fired.

zc.intid.subscribers.removeIntIdSubscriber(ob, event)[source]

Removes the unique ids registered for the object in all the unique id utilities.

Just before this happens (for the first time), an zc.intid.interfaces.IBeforeIdRemovedEvent is fired, followed by an zope.intid.interfaces.IIntIdRemovedEvent. Notice that this is fired before the id is actually removed from any utility, giving other subscribers time to do their cleanup.

Before each utility removes its registration, it will fire zc.intid.interfaces.IIdRemovedEvent. This gives a guaranteed order such that zope.catalog and other Zope event listeners will have fired.

zc.intid.subscribers.intIdEventNotify(event)[source]

Event subscriber to dispatch IntIdEvent to interested adapters.

See subscribers.zcml for its registrations (it handles two types of events).

Changes

2.0.1 (unreleased)

  • Nothing changed yet.

2.0.0 (2016-12-16)

  • Add zope.lifecycleevent subscribers. You must include subscribers.zcml to use these and have zope.intid installed. See issue #5.
  • Documentation is now hosted at http://zcintid.readthedocs.io
  • Add continuous integration testing for supported Python versions.
  • Add PyPy support.
  • Add Python 3 support.
  • Drop support for Python less than 2.7.
  • Remove ZODB3 dependency in favor of explicit dependencies on BTrees.
  • The zope-intid.zcml file included in this package now works to make the IntId utility from this package implement the zope.intids interface, if that package is installed.
  • Interfaces and event implementations have been refactored into the new module zc.intid.interfaces. Backwards compatibility aliases remain for the old names. See issue #9.
  • Raise more informative KeyError subclasses from the utility when intids or objects cannot be found. This distinguishes them from errors raised by normal dictionaries or BTrees, and is useful in unit testing or when persisting intids or sharing them among processes for later or concurrent use. See issue #8
  • Propagate POSKeyError from queryId instead of returning the default object. This exception indicates a corrupt database, not a missing object. The queryObject function already behaved this way.
  • Attempting to register an object that cannot have the utility’s attribute set on it (for example, it has restrictive __slots__) no longer corrupts the utility’s state.

1.0.1 (2011-06-27)

  • Make the behavior of the utility’s getId method consistent with zope.intid in regard to its handling of proxied objects.

1.0.0 (2011-02-21)

  • Initial release.

Development

zc.intid is hosted at GitHub:

Project URLs

Indices and tables