Daemon interface

A daemon is the source of authority for one or more mKTL items; when a client gets or sets an item value, that request is handled by the daemon. Exactly how that request gets handled is entirely up to the daemon code; there are no restrictions imposed by the mKTL protocol itself, it is up to the daemon to resolve any consistency issues or conflicting requests.

The default behavior of the mktl.Store is to act as a caching key/value store: values come in, values go out, they may or may not persist across restarts depending on the configuration. This behavior is rarely sufficient for an application; a typical daemon will implement custom code behind the getting and setting of individual items. The Python interface described here expects the user to create custom subclasses of mktl.Daemon, which in turn may instantiate custom subclasses of mktl.Item. Background threads are used widely, polling and callbacks in particular are two areas where dedicated per-item background threads will invoke any/all registered methods; the author of any custom code should not assume that calls arriving via these mechanisms will be serialized or thread-safe to any meaningful degree.

The Daemon class

The mktl.Daemon class is the entry point and overall organization for the daemon; while it provides a common storage location for daemon-wide functions, such as handling requests and publishing broadcasts, from the perspective of the developer the mktl.Store is a container for the mktl.Item instances that define the actual behavior of the system. An incoming set request, for example, is not handled by the any higher level construct, it is passed to the appropriate mktl.Item instance for proper handling.

The mktl.Daemon also defines the initialization sequence, if any, for the daemon: which steps to take upon startup, which mktl.Item subclasses to invoke for specific items, whether other initialization steps must occur such as defining a common communication point to access a hardware controller, or whether the initial state of a set of items needs to be manipulated before proceeding with routine operations.

In nearly all cases the developer will need to create a custom mktl.Daemon subclass to satisfy the operational goals of an individual daemon.

class mktl.Daemon(store, alias, override=False, arguments=None)

The mKTL Daemon is a facilitator for common mKTL actions taken in a daemon context: loading a configuration file, instantiating mktl.Item instances, and commencing routine operations.

The developer is expected to subclass the Daemon class and implement a setup() method, and/or a setup_final() method. This subclass is the gateway between mKTL functionality and the domain-specific custom code appropriate for the developer’s application.

The store argument is the name of the store that this daemon is providing items for; alias is the unique name of this mKTL daemon, and is used to locate the configuration file that defines the items for which this daemon is authoritative.

arguments is expected to be an argparse.ArgumentParser instance, though in practice it can be any Python object with specific named attributes of interest to a Daemon subclass; the arguments argument is not required. This is intended to be a vehicle for custom subclasses to receive key information from command-line arguments, such as the location of an auxiliary configuration file containing information about a hardware controller.

add_item(item_class, key, **kwargs): Add an mktl.Item to this daemon instance; this is the entry point for establishing an authoritative item, one that will handle inbound get/set request and the like. The kwargs will be passed directly to the item_class when it is called to be instantiated.

cleanup(*args, **kwargs): Subclasses should override the cleanup() method to perform any/all actions prior to shutting down the daemon, such as publishing a “shutting down” message, or cleanly terminating a connection with a hardware device. This method is guaranteed to only be invoked a single time. The default implementation of this method takes no actions.

setup(): Subclasses should override the setup() method to invoke add_item() for any custom mktl.Item subclasses or otherwise execute custom code. When setup() is called the bulk of the Daemon machinery is in place, but cached values have not been loaded, nor has the presence of this daemon been announced. The default implementation of this method takes no actions.

setup_final(): Subclasses should override the setup_final() method to execute any/all code that should occur after all mktl.Item instances have been created, including any non-custom mktl.Item instances, but before this daemon announces its availability on the local network. The default implementation of this method takes no actions.

stop(): Request that this daemon stop execution.

The Item class, expanded

The bulk of any daemon-specific logic will occur in mktl.Item subclasses. This is where requests get handled, where data gets interpreted, where logic is defined that could span items within and without the boundaries of the containing mktl.Daemon.

The key methods to override when implementing custom behavior are mktl.Item.perform_get(), mktl.Item.perform_set(), and mktl.Item.validate(). In addition, mktl.Item.req_poll() will be of interest, used in combination with mktl.Item.register(), and the mktl.Item.value property, as well as the mktl.Item.formatted and mktl.Item.quantity variants, to explicitly set a new item value.

class mktl.Item(store, key, subscribe=True, authoritative=False, pub=None)

An Item represents a key/value pair, where the key is the name of the Item, and the value is whatever is provided by the authoritative daemon. The principal way for both clients and daemons to get or set the value is via the value() property.

A non-authoritative Item will automatically subscribe() itself to any available updates.

from_payload(payload)

Recreate the fundamental Python type of the value from the provided mktl.protocol.message.Payload instance, and return it to the caller.

This default handler will interpret the bulk component, if any, of the payload as an N-dimensional numpy array, with the description of the array present in the other fields of the payload.

This is the inverse of to_payload().

perform_get()

Acquire the most up-to-date value available for this Item and return it to the caller. Return None if no new value is available; if a mktl.protocol.message.Payload instance is returned it will be used as-is, otherwise the return value will be passed to to_payload() for encapsulation.

Returning None will not clear the currently known value, that will only occur if the returned Payload instance is assigned None as the ‘value’; this is not expected to be a common occurrence, but if a custom perform_get() implementation wants that to occur they need to instantiate and return the Payload instance directly rather than use to_payload().

perform_set(new_value)

Implement any custom behavior that should occur as a result of a set request for this item. No return value is expected. Any subclass implementations should raise an exception in order to trigger an error response.

Any return value, though again none is expected, will be encapsulated via to_payload(), after the same fashion as perform_get(), and included in the response to the original request.

poll(period): Poll for a new value every period seconds. Polling will be discontinued if period is set to None or zero. Polling will invoke req_poll(), and occurs at the requested interval within a background thread unique to this item.

publish(new_value, timestamp=None, repeat=False)

Publish a new value, which is expected to be the Python native representation of the new value. If timestamp is set it is expected to be a UNIX epoch timestamp; the current time will be used if it is not provided. Newly published values are always cached locally.

Note that, for simple cases, an authoritative daemon can set the value() property to publish a new value instead of calling publish() directly. In other words, these two calls are equivalent:

item.value = new_value
item.publish(new_value)

req_get(request): Handle a GET request. The request argument is a protocol.message.Request instance; the value returned from req_get() is identical to the value returned by perform_get(), which is where custom handling by subclasses is expected to occur.

req_poll(repeat=False)

Handle a background poll request, established by calling poll(). perform_get() is where custom handling by subclasses is expected to occur. The payload returned from req_poll() is identical to the payload returned by perform_get().

A common pattern for custom subclasses involves registering req_poll() as a callback on other items, so that the value of this item can be refreshed when external events occur.

req_set(request)

Handle a SET request. The request argument is a protocol.message.Request instance; the value returned from req_set() will be returned to the caller, though no such return value is expected. Any calls to req_set() are expected to block until completion. Custom handling by subclasses is expected to occur in perform_set().

If the publish_on_set attribute is set to True (this is the default) a call to publish() will occur at the tail end of any successful SET request. Custom subclasses can set this attribute to False to inhibit that behavior.

to_payload(value=None, timestamp=None)

Interpret the provided arguments into a mktl.protocol.message.Payload instance; if the value is not specified the current value of this Item will be used; if the timestamp is not specified the current time will be used. This is particularly important as a step in a custom perform_get() implementation.

This is the inverse of from_payload().

validate(value): A hook for a daemon to validate a new value. The default behavior is a no-op; any checks should raise exceptions if they encounter a problem with the incoming value. The ‘validated’ value should be returned by this method; this allows for the possibility that the incoming value has been translated to a more acceptable format, for example, converting the string ‘123’ to the integer 123 for a numeric item type.

mkd executable

The mkd executable provides a command-line interface to invoke a persistent daemon executing a mktl.Daemon subclass as described above. It is the natural starting point for any persistent daemon implementing the scheme described in this document, since that is the precise purpose it is written for.

Refer to the section covering mkd in the Executables documentation for additional details.