Storage Plugin Tutorial

Hash needs to store its resources files and state in a storage system such as disk, buckets, database, etc… We have built the storage system to be easily configurable to change from one storage backend to another, we defined a clear API for the system, which can be implemented for different backends, we are using pluggy to give the developers the ability to write plugins for extending the storage backends supported, even the internal hash storage systems are implemented using plugins.

Now let us talk about the API for any storage plugin:

init

This method must use store_hookimpl and store_init dectorators is used to intialize the plugin, here we pass a dictionary with the configuration options defined for the plugin in config.ini file and do any configuration needed, this method must raise the ConfigStoreError if there is anything wrong with configuration.

@store_hookimpl
@store_init
def init(self, config):
    # Read and check config for required parameters for your plugin
    pass

store_artifact

This method takes the artifact’s object and resource uid as arguments, and writes the artifact’s file to the storage backend, only file artifacts are saved using this method, other artifacts are saved in the state’s object. This method must also save the relative paths to artifacts in the backend, these will be needed later for artifact_paths method, make sure these paths are unique.

artifact_paths

This method takes a resource ID as its parameter, and returns a list of relative paths for artifacts for this resource.

read_artifact

This method takes the artifact’s object and resource uid as arguments, and writes the artifact’s file to the resource’s directory with the same name as it was before, it’s used to read artifact files from state.

get_states

This method takes resource’s uid as argument and returns a dictionary of all the states for this resource, the hash is used as dictionary keys.

get_globals

This method returns a dictionary of global vlaues stored in the storage system, these globals are generated from running actions on other resources.

store_globals

This method takes globals dictionary as an argument and saves it to the storage backend, it can safely overwrite the stored globals because the argument is guranteed to be complete for all global values.

store_state

This method takes state’s dictionary and resource uid as arguments and stores them in the storage backend, the states’ dictionary is guranteed to be complete and you can safely overwrite old states.