Introduction

Pangolin Plugin Manager is a software to easily develop plugins for with a already existing permission / role framework of Pangolin which can be easily worked with by the installed plugins.

Plugins are fully independend pieces of software within Pangolin and can perform their own logic and are loaded when starting pangolin.

Features

User Permission Management

Each installed plugin can define it's own permissions which are exposed at the Management -> Users page for users with the role pangolin:admin or pangolin:manager-users.

In this example the installed plugin rNums (random-Numbers) has two permissions, configure and access which can be selectively given to each user if necessary.

Plugin Administration

As this application can be scaled up with custom plugins it was necessary to give a way to perform certain elementary actions on them such as start and stop. The select action will display the logs of the plugin. By default this screen is accessible by users with either the role pangolin:admin or pangolin:manager-scaling.

Plugins themselves can also individually allow or disallow users those actions, in this case rNums gives access to those actions for users who have the rNums:configure role.

Demonstration

Example of rNums Plugin (randomNumbers)

Example: Stop

Example: Permission (access)

Architecture Decision Record: Plugin Execution Strategy

Context

Various methods to run plugins were evaluated, including:

1. Direct injection into the Node.js runtime
2. Execution as separate threads
3. Execution as child processes

Each approach has its own advantages and disadvantages, as outlined below.

Considered Options

1. Direct Injection into Node.js Runtime

• Advantages:
  • Simple to implement.
  • Can enforce a specific interface, ensuring all plugins adhere to a predefined schema.
• Disadvantages:
  • Direct access to the runtime poses security risks.
  • Lack of isolation means plugin failures could affect the entire system.

2. Execution as Threads

• Advantages:
  • Faster communication between the main application and the plugin, as threads share memory space.
  • Moderate isolation, where failures within a thread are more manageable than with direct injection.
  • Lower overhead compared to child processes.
• Disadvantages:
  • Some overhead due to thread management.
  • Less isolation compared to child processes, so plugin errors can potentially impact the main application.
  • Communication with Threads is not troublesome and takes latency and lots of overhead which is not ideal.

3. Execution as Child Processes

• Advantages:
  • Complete isolation from the main application, providing enhanced stability and crash protection.
  • Easier monitoring and control; plugin crashes do not affect the core application.
• Disadvantages:
  • Data exchange between the main application and the child process is more complex and less reliable.
  • No guarantee that the plugin will meet all functional requirements, as it operates in a separate process without a predefined schema.

Decision

A lot of time was invested into running the plugins as child processes, but it was just exponentially more effort for only a few advantages. And as plugins are already designed to be as permissive as possible, they are expected to be trusted and reliable in what they do, so I would argue that child processes with their additional isolation are not needed.

The decision is to implement plugin execution using threads. This approach offers a balanced solution by providing faster communication with lower overhead while maintaining a degree of isolation and crash protection. It combines the benefits of direct injection and child processes, making it a suitable compromise for this context.

Example Code: Directly Injected vs Child Processes

Here is a equal code implemented in two different ways to show the downside of child process and the difficulty it brings in handling data exchange between processes.

Injected:

router.get("/api/admin/plugin/:name/start", (req: Request, res: Response) => {
    const name = req.params.name;
    const plugin = manager.getPlugin(name);
    const perms: Permissions = req.permission;

    if (plugin == null) {
        res.send({"error": "no plugin"});
    } else if (plugin.isConfigurable(perms) || perms.hasAnyPermission(manager.PERMISSIONS.ADMIN, manager.PERMISSIONS.MANAGER_SCALING)) {
        plugin.start();
        res.send({"status": plugin.getState()});
    } else {
        res.send({error: "unauthorized"});
    }
});

Child Process:

router.get("/api/admin/plugin/:name/start", (req: Request, res: Response) => {
    const name = req.params.name;
    const pluginProcess = PluginManager.getPluginProcess(name);  // Get the child process of the plugin
    const perms: Permissions = req.permission;

    if (!pluginProcess) {
        return res.status(404).send({ "error": "Plugin not found" });
    }

    // Check if user has the necessary permissions to start the plugin
    if (!perms.hasAnyPermission(PluginManager.PERMISSIONS.ADMIN, PluginManager.PERMISSIONS.MANAGER_SCALING)) {
        return res.status(403).send({ error: "Unauthorized" });
    }

    // Send a message to the plugin process along with the user's permissions
    pluginProcess.send({
        type: 'startPlugin',
        permissions: perms.getPermissions()  // Send the permissions to the plugin process
    });

    // Listen for the plugin's response
    pluginProcess.once('message', (message: any) => {
        if (message.type === 'pluginState') {
            res.send({ status: message.state });
        } else {
            res.status(500).send({ error: 'Failed to start plugin' });
        }
    });

    // Handle plugin process errors
    pluginProcess.once('error', (error: any) => {
        console.error(`Error in plugin ${name}:`, error);
        res.status(500).send({ error: 'Plugin process error' });
    });
});

Implementation Details

Terminology

Refering to the core means the main pangolin application.

Plugins, or in this context also sometimes refered to as "scales" (due to the Pangolin animal), are modules that can be installed and fully managed in the pangolin web panel.

Plugin API

Plugins are modules that can be installed into the core pangolin application. They require a index.ts file which implements the following typescript interface:

export interface Plugin {
    name: string;    // Plugin name
    version: string; // Plugin version

    initialize(): void; // Logic for initiliazing a plugin once
    start(): void; // Logic for starting a plugin
    stop(): void; // Logic for stopping a plugin

    getPermissions(): string[]; // String-Array of available permissions
    getState(): string; // Get state of plugin
    getLogs(): string[]; // Get logs of plugin

    // Check if user can access/view this plugin
    isAccessible(permissions: Permissions): boolean; 

    // Check if user can manage this plugin (start, stop)
    isConfigurable(permissions: Permissions): boolean; 

    // Register http endpoints for the /plugin/<pluginname>/ path
    registerEndpoints(router: Router): void; 
}

This approach gives full flexibility towards the plugin, but it also comes with a lot of responsibility. For example, the core has to trust the plugin to return the correct state when calling plugin.getState().

However, the benefit is that the plugins can act as their own full application, in theory even allowing a pangolin plugin in pangolin. The router given for the registerEndpoints() function is for the path /plugin/, and they have full control over their routes and what they want to display.

Plugin Loading

The plugins are simply being loaded by iterating through the /scales directory and and requiring the subdirectories, and storing the instances in a Map with their plugin name as a key to keep them accessible.

Permission System

Permission Parser (Middleware)

The express.js request interface has been expanded to include a request.permission which returns a Permission Object, which gets injected with a permission-parser interface on each incoming request. A user is identified by a random generated UUIDv4 which acts as a token.

export function permissionParser(req: Request, res: Response, next: NextFunction) {
    const token = req.cookies?.token;

    if (!token) {
        req.permission = new Permissions(null, []);
        next();
        return;
    }

    const userid : string | null  = auth.getUserIdByToken(token);
    if (userid) {
        req.permission = new Permissions(userid, auth.getUserPermissions(userid));
    } else {
        req.permission = new Permissions(null, []);
        res.clearCookie("token");
    }

    next();
}

Permission Object

As described earlier, the permission object from the express.js request provides functions to check for permissions from an incoming request, and is implemented like this:

export default class Permissions {
    private permissions: string[];
    private userid: string | null;

    constructor(userid: string | null, permissions: string[]) {
        this.permissions = permissions;
        this.userid = userid;
    }

    public getPermissions(): string[] {
        return this.permissions;
    }

    // Method to check if a single permission exists
    public hasPermission(permission: string): boolean {
        return this.permissions.includes(permission);
    }

    // Method to check if all permissions are granted
    public hasAllPermissions(...permissions: string[]): boolean {
        return permissions.every(permission => this.permissions.includes(permission));
    }

    // Method to check if any of the given permissions are granted
    public hasAnyPermission(...permissions: string[]): boolean {
        return permissions.some(permission => this.permissions.includes(permission));
    }

    // True if session has no token or no token that can be associated with a user.
    public isGuest(): boolean {
        return this.userid == null;
    }
}

With this plugins can access the permissions of an incoming request and and permit or deny them the access to the resource. This solution provides a centralized permission system from the core for otherwise independend plugins.

Permissions

By default there are following permissions available:

• pangolin:admin permits access to all pangolin tools
• pangolin:manager permits access to pangolin management page
• pangolin:manager-scales permits configuration access to all plugins
• pangolin:manager-users permits to configure all users and their roles

All plugins can have as many own permissions as they desire, however they need to start with <plugin_name>:

Example Plugin "rNums" (Random Numbers)

It's a plugin to demonstrate that, in theory, there are no limitations to the plugins. This plugin runs a python script as a child process, which generates 5 random numbers each second and stores them in a numbers.json database.

rNums also registers its own frontend using the website template to have a seemless integrated look, which simply displays the 5 generated numbers in a table. This works by also registering an api endpoint for its own web path (GET /data) which returns the json database. This data is being fetched each second from the frontend using javascript and being displayed.

Just visiting the rNums page might seem like it's a meaningless plugin, but it proves that the plugins can fully implement their own logic (python script for generating random numbers), store persistent data, and implement their own frontend and access their data from here.

Future Plans

Plugin Config

In the admin panel settings tab add a configuration interface for each plugin, where also users who are permitted by isConfigurable() can configure parameters for the plugin.

Plugin Deployment

Allow the users with pangolin:manager-scales permission to deploy and undeploy plugins in runtime.

Conflict Avoidance

When a new plugin is being deployed verify that there are no conflicts with existing plugins, such as:

1. Duplicate Plugin Names Could be fixed by keeping track of all current plugins and their respective names.

2. Duplicate Permission Names Could be fixed by forcing plugins to name permissions in a specific format, such as starting with <pluginname>:

Wrapped Plugin Files

It would be a fun topic to learn more about using my own file format that wrap / encapsulate the plugin, to deploy a singular file and not an entire folder (like .EAR or .WAR files). I would need to come up with a solution for storing persistent data, and if it makes sense to store it in the plugin file or somewhere else.

Plugin Statuspage

Theoretically, a statuspage where metrics about performance usage are provided would be pretty cool, however the current approach to loading the plugins would need to be changed, and they would be needed to run as subprocesses each, and not injected into the current nodejs process.

Written: 2024-09-20