File API

A task can write (for outputs) or read (for inputs) binary files.

Background

Note that in the JSON representation of inputs and outputs, we do not want to encode the binary file content, wile in theory possible. The idea of the File API is to only transmit references to binary files in these documents.

The most natural representation of a reference to a binary file would be a simple file path. Of course, it is possible to just pass such a file path in the form of a string in the input / output of a task. However, keep in mind that the typical use-case of HQS Tasks plans to execute the task not on the same machine where the client is running, but instead on some remote machine. Also, when invoking a follow-up task, which shall read the file written by the previous task, just passing over the path of a local file is also not working, since the tasks are executed in an isolated, containerized environment and therefore do not share any file system in general.

So when it comes to exchanging data via files it's important that also the file itself is transmitted, not just the path. HQS Tasks does exactly that automatically for you when you tell it that the string is a file path. This is triggered by having the file path wrapped in a JSON object with the special key $file. To represent such a JSON document, hqs_task provides a dedicated FileRef type.

Writing Files (Output)

When a task has written a file as part of its implementation, it can tell HQS Tasks that this file is meant to be made available to the caller by including a FileRef value in the return value (usually by adding a field of type FileRef to a model which is the return type or part of it).

Simple example, where the whole output is just a FileRef:

from hqs_task import task
from hqs_task.types.fileref import FileRef

@task
def write_file() -> FileRef:
    local_filename = "file.txt"
    with open(local_filename, "wt") as f:
        f.write("Hello world!")

    return FileRef(local_filename)

Reading Files (Input)

When a task wants to receive a file as part of its input, it can tell HQS Tasks that by including a FileRef value in (one of) the arguments (by using FileRef as the argument's type or by adding a field of type FileRef to a model used in the argument type).

Simple example, where the whole input (since the function is just taking a single argument) is just a FileRef:

from hqs_task import task
from hqs_task.types.fileref import FileRef

@task
def read_file(file: FileRef) -> None:
    with open(file.filename, "rt") as f:
        print(f.read())