Skip to content

Running tasks on remote machines using SSH🔗

The bandsaw.advices.ssh.SshAdvice allows to run tasks on a different machine than where the workflow defining the whole process is being run. Internally it uses the ssh command line tools to transfer the task over to another machine, spawns a new python interpreter, runs the task and returns the result back to the calling machine.

Configuration🔗

directory (str)🔗

The directory, where the temporary files like the session or the result will be stored before transferring them to or from a remote machine. If the directory doesn't exist, it will be created when first being used. If no directory is given, a temporary directory is used.

add_remote()🔗

Defining the remote machine is done by calling add_remote(remote) on the advice and providing an instance of the bandsaw.advices.ssh.Remote class. The class contains all the information necessary for executing a task remotely and is initialized using its constructor:

host🔗

The hostname of the machine, where the tasks should be run. It could be an ip address, too, but it must be set.

port🔗

The port to which SSH should connect to as int. If not set, the default port 22 for ssh is used.

key_file🔗

key_file should be the path to file, which contains the key that will be used for authentication. If this is not configured, ssh will use the default.

Warning

Due to its automatic nature, bandsaw doesn't know how to handle passwords. Therefore, it doesn't support authentication via password or keyfiles that use a passphrase (without using a ssh-agent).

user🔗

The username of the user, that is being used on the remote machine. If this is not configured, bandsaw uses the name of the local user, that runs the python interpreter.

interpreter🔗

An instance of bandsaw.interpreter.Interpreter which contains the path on the remote machine to a python interpreter as well as additions to the PYTHONPATH on the remote machine, e.g. if non-standard libraries are used.

directory🔗

Path to a directory on the remote machine, where temporary files will be stored. If not specified, '/tmp' is as default.

Example configuration with a single remote🔗

import bandsaw
from bandsaw.advices.ssh import SshAdvice, Remote

configuration = bandsaw.Configuration().add_advice_chain(
    SshAdvice().add_remote(
        Remote(
            host='my.remote.machine.tld',
            port='22',
            key_file='/path/to/my/key',
            user='my_remote_user',
            interpreter=bandsaw.Interpreter(
                path=[],
                executable='/my/remote/path/to/python3',
            ),
            directory='/my/remote/directory',
        ),
    ),
)

Example configuration with multiple remotes🔗

The advice supports multiple different remote machines that can be used for executing tasks. Each definition of a remote machine has to be added by a add_remote(remote) call to the SshAdvice class:

import bandsaw
from bandsaw.advices.ssh import SshAdvice, Remote

configuration = bandsaw.Configuration().add_advice_chain(
    SshAdvice().add_remote(
        Remote(
            host='my.remote.machine.tld',
            port='22',
            key_file='/path/to/my/key',
            user='my_remote_user',
            interpreter=bandsaw.Interpreter(
                path=[],
                executable='/my/remote/path/to/python3',
            ),
            directory='/my/remote/directory',
        ),
        name='remote-1',
    ).add_remote(
        Remote(
            host='my.other.machine.tld',
            port='2222',
            key_file='/different/key',
            user='my_other_user',
            interpreter=bandsaw.Interpreter(
                path=[],
                executable='/my/remote/path/to/python3',
            ),
            directory='/tmp/directory',
        ),
        name='remote-2',
    ),
)

Besides the Remote instance, the add_remote() method takes an additional argument name which allows referring to a specific remote configuration. If no name is given 'default' is used.

Which remote configuration should be used can be configured per task. Each task definition can take additional task arguments in form of keyword arguments:

@bandsaw.task(ssh={'remote': 'remote-2'})
def greet(recipient):
    return f"Hello {recipient}"

The SshAdvice expects a dictionary for the task keyword argument ssh. This dict contains then the task specific SSH configuration. Currently, the only supported key is 'remote' whose value must be the name of one of the remotes defined by add_remote(). If no task keyword argument 'ssh' is given, or its value doesn't contain 'remote', the default name for the remote is 'default'.


Last update: 2021-11-11