Plugins Architecture Concepts

Overview

Plugins are dynamic shared libraries (.so files in Unix, .dll files in Windows) that export C calling convention functions. Programs like Falco dynamically load these libraries and call the exported functions to extend Falco's support for event sources/fields.

Plugins are versioned using semantic versioning to minimize regressions and compatibility issues.

Plugins can be written in any language, as long as they export the required functions. Go, however, is the preferred language to write plugins, followed by C/C++.

Plugins can implement one or more capabilities. In the scope of plugins, a capability is an extension of Falco's features in the form of a specific set of C function symbols exported by shared libraries. Currently, there are four plugin capabilities supported by the framework: event sourcing, field extraction, event parsing and async event

Plugins are Coresident with Falco

The libraries will do everything possible to validate the data coming from the plugins and protect Falco and the other consumers from corrupted data. However, for performance reasons, plugins are "trusted": they run in the same thread and address space as Falco and they could crash the program. We assume that the user will be in control of plugin loading and will make sure only trusted plugins are loaded/packaged with Falco.

Plugin SDKs

To make it easier to write plugins, there are Go and C++ SDKs that handle the details of memory management and type conversion. These SDKs provide a streamlined way to implement plugins without having to deal with all the details of the lower-level functions that make up the Plugin API.

These SDKs are optional, but using them is highly recommended.

Event Sourcing Capability

Plugins with event sourcing capability provide a new event source and make it available to libscap and libsinsp. They have the ability to "open" and "close" a stream of events and return those events to the plugin framework. They also provide a plugin ID, which is globally unique and is used in capture files (see below). Event sources provided by plugins with this capability are tied to the events they generate and can be used by plugins with field extraction capabilities and within Falco rules (see below).

Field Extraction Capability

Plugins with field extraction capability have the ability to extract information from events based on fields. For example, a field (e.g. proc.name) extracts a value (e.g. process name like nginx) from a syscall event. The plugin returns a set of supported fields, and there are functions to extract a value given an event and field. The plugin framework can then build filtering expressions (e.g. rule conditions) based on these fields combined with relational and/or logical operators. For example, given the expression ct.name=root and ct.region=us-east-1, the plugin framework handles parsing the expression, calling the plugin to extract values for fields ct.name/ct.region for a given event, and determining the result of the expression. In a Falco output string like An EC2 Node was created (name=%ct.name region=%ct.region), the plugin framework handles parsing the output string, calling the plugin to extract values for fields, and building the resolved string, replacing the template field names (e.g. %ct.region) with values (e.g. us-east-1).

Plugins with this capability only focus on field extraction from events generated by other plugins or by the core libraries. They do not provide an event source but can extract fields from other event sources. The supported field extraction can be generic or be tied to a specific event source. An example is json field extraction, where a plugin might be able to extract fields from generic json payloads.

Event Parsing Capability

Plugins with event parsing capability can hook into an event stream and receive all of its events sequentially. The parsing phase is the stage in the event processing loop in which the Falcosecurity libraries inspect the content of the events' payload and use it to apply internal state updates or implement additional logic. This phase happens before any field extraction for a given event. Each event in a given stream is guaranteed to be received at most once.

Async Events Capability

Plugins with async events capability can enrich an event stream from a given source (not necessarily implemented by itself) by injecting events asynchronously in the stream. Such a feature can be used for implementing notification systems or recording state transitions in the event-driven model of the Falcosecurity libraries, so that they can be available to other components at runtime or when the event stream is replayed through a capture file.

For example, the Falcosecurity libraries leverage this feature internally to implement metadata enrichment systems such as the one relative to container runtimes. In that case, the libraries implement asynchronous jobs responsible of retrieving such information externally outside of the main event processing loop so that it's non-blocking. The worker jobs produce a notification event every time a new container is detected and inject it asynchronously in the system event stream to be later processed for state updates and for evaluating Falco rules.

Composability of Capabilities

Plugin capabilities are composable, meaning that a single plugin can implement one or more capabilities. At loading time, the framework is able to recognize which capabilities the plugin correctly implements, and uses it accordingly inside Falco and the libraries. There can be plugins implementing event sourcing only, field extraction only, or both. For example, the AWS CloudTrail plugin implements both capabilities in order to provide the framework with the aws_cloudtrail event source and to allow extracting fields such as ct.name from the events it produces.

Plugin Event IDs

Every plugin with event sourcing capability requires its own, unique plugin event ID to interoperate with Falco and the other plugins. This ID is used in the following ways:

• The ID is saved in in-memory event objects and is used to identify the associated plugin that injected the event.
• The ID is saved in capture files and is used to recreate in-memory event objects when reading capture files.

The ID must be unique to ensure that events written by a given plugin will be properly associated with that plugin (and its event sources, see below).

Plugin authors must register the plugin with the Falcosecurity organization by creating a PR to modify the plugin registry with details on the new plugin. This ensures that a given ID is used by exactly one plugin.

Plugin Event Sources and Interoperability

Events returned by plugins with event sourcing capability have an event source that describes the event's information. This is distinct from the plugin name to allow for multiple plugin implementations to generate the same kind of events. For example, there might be plugins gke-k8saudit, eks-k8saudit, ibmcloud-k8saudit, etc. that all fetch K8s Audit information. The plugins would have different names and IDs but would have the same event source k8s_audit.

A plugin with field extraction capability optionally provides a set of compatible event sources. When the framework receives an event with an event source in the plugin's set of event sources, fields in expressions (i.e., fields included in the rule's output field) will be extracted from events using the plugin. The set of compatible event sources can also be omitted. In this case, all events will be presented to the plugin, regardless of their source. In this case, the plugin must detect the format of arbitrary payloads and be able to return NULL/no value when the payload is not supported. As such, given a specific event source such as k8s_audit, there is an implicit contract to honor regarding how data is formatted in each event of that source, such that compatible plugins with field extraction capability are able to parse events of a certain source even if they are produced by different plugins.

Plugin authors should register the plugin with the Falcosecurity organization by creating a PR to modify the plugin registry file with details on the new plugin. This allows plugin authors to coordinate about event source data formats.

Handling Duplicate/Overlapping Fields in Plugins/Libraries Core

At an initial glance, adding plugins introduces the possibility of tens/hundreds of new filtercheck fields that could potentially overlap/conflict. For example, what happens if a plugin defines a proc.name field? However, the notion of event source makes these potential conflicts manageable.

Remember that field extraction is always done in the context of an event, and each event can be mapped back to an event source. So we only need to ensure that filtercheck fields are non-overlapping for a given event source. For example, it's perfectly valid for an AWS CloudTrail plugin to define a proc.name field, as the events generated by that plugin are wholly separate from syscall events. For syscall events, the AWS CloudTrail plugin is not involved and the core libraries extract the process name for the tid performing a syscall. For AWS CloudTrail events, the core libraries are not involved in field extraction. Extraction is performed by the AWS CloudTrail plugin instead.

When managing plugins, we only need to ensure the following:

• That only one plugin is loaded at a time that exports a given event source. For example, the libraries can load either a gke-k8saudit plugin with event source k8s_audit, or eks-k8saudit with event source k8s_audit, but not both.
• That for a mix of plugins with event sourcing and field extraction capabilities having the same event source, that the fields are distinct. For example, a plugin event sourcing capability providing the source k8s_audit can export ka.* fields, and a plugin with field extractor capabilities with event source k8s_audit can export a jevt.value[/...] field, and the appropriate plugin will be used to extract fields from k8s_audit events as fields are parsed from condition expressions/output format strings.

Plugin API

Here is an overview of the functions that comprise the plugin API. This list is not extensive: the plugin API reference has full documentation of plugin APIs for all the capabilities supported by the framework.

In almost all cases, a plugin author can use the SDKs which provide a more streamlined interface. This still provides a good overview of the functionality a plugin provides.

Info Functions

A set of functions provide information about the plugin and its compatibility with the plugin framework:

• plugin_get_required_api_version: Return the version of the plugin API used by a plugin.
• plugin_get_name: Return the name of the plugin.
• plugin_get_description: Return a short description of the plugin.
• plugin_get_contact: Return a contact url/email/twitter account for the plugin authors.
• plugin_get_version: Return the version of the plugin itself.
• plugin_get_last_error: Return the error that was last generated by a plugin.
• plugin_get_id: Return the unique ID of the plugin (event sourcing capability only).
• plugin_get_event_source: Return a string describing the events generated by a plugin (event sourcing capability only).
• plugin_get_async_events: Return a list of async events produced by a plugin (event parsing capability only).
• plugin_get_init_schema: (Optional) Return a string describing a schema for the configuration passed to plugin_init.
• plugin_list_open_params: (Optional) Return a list of suggested valid parameter values for plugin_open.
• plugin_get_extract_event_sources: (Optional) Return a list of all compatible event sources (field extraction capability only).
• plugin_get_extract_event_types: (Optional) Return a list of all compatible event types (field extraction capability only).
• plugin_get_parse_event_sources: (Optional) Return a list of all compatible event sources (event parsing capability only).
• plugin_get_parse_event_types: (Optional) Return a list of all compatible event types (event parsing capability only).
• plugin_get_async_event_sources: (Optional) Return a list of all compatible event sources (async events capability only).

Instance/Capture Management Functions

Plugins have functions to initialize/destroy a plugin, as well as functions to open/close streams of events:

• plugin_init: Initialize the plugin and, if needed, allocate its state.
• plugin_destroy: Destroy the plugin and, if plugin state was allocated, free it.
• plugin_open: Open the source and start a stream of events (event sourcing capability only).
• plugin_close: Close a stream of events (event sourcing capability only).

Events/Fields Related Functions

Plugins with event sourcing capability have functions to provide events to the plugin framework:

• plugin_next_batch: Return one or more events to the plugin framework.
• plugin_get_progress: (Optional) Provide feedback on how much of the event stream has been read.
• plugin_event_to_string: (Optional) Return a text representation of an event generated by a plugin.

Plugins with field extraction capability have functions to define the set of fields that can be used to extract information from events, to actually extract values from events, and to return printable representations of events:

• plugin_get_fields: Return the list of extractor fields exported by a plugin.
• plugin_extract_fields: Extract one or more filter field values from an event.

Other capabilities allow interacting with events for different scopes functionalities:

• plugin_parse_event: Parses an event at most once before the extraction phase, and can perform state updates.
• plugin_set_async_event_handler: Registers a callback that can be used to inject asynchronous events in an open event stream.