Error Reporting

As with every software program, a task may encounter an error case. We distinguish between different kinds of errors:

  1. Invalid input: The task can not run with the given input. We furthermore distinguish between the situations:
    • Input is not a valid instance of the input schema (a.k.a. "validation error")
    • Input is a valid instance of the input schema, i.e., the schema did not specify that this input is invalid. This is a very reasonable scenario as it is unfeasible to reflect every corner case (of what is valid and what is not) in the schema. An example is that the product of two numbers in the input must not exceed a specific value. We also call this a "extended validation" which failed in this case.
  2. (In principle valid but) Unprocessable input: The task is designed to handle the given input in principle, and also attempted to process it, but ran into the situation that it can't process it further. We furthermore distinguish:
    • The task detects the situation and writes an error report. This case also includes raised exceptions (as they will be translated to error reports, see below) in cases the software developer did not fully design the task for, and most "simple bugs".
    • The task crashes without writing an error report.
  3. (In principle valid and processable but) Unsolvable input: The task can process the given input in principle, but ran into the situation that a final solution could not be found. Depending on the subject, this might not be considered an error, but more like a "negative outcome", an example being an interative algorithm to not converge into a stable solution after many iterations.

Beyond these scenarios, in the overall HQS Tasks system, there are even more situations where we can run into exceptional cases, which we do not discuss here. An example being when the client tries to invoke a task that does not exist.

Validation Errors

t.b.d.

Error Reports (a.k.a. Exceptions)

Before diving deeper, let us first clarify that on the CLI level of a task we talk about "error reports", while in most programming languages these correspond to "exceptions": In HQS Tasks, we define an error report independent of a programming language, and then map these to exceptions in the programming languages being used.

An error report is a JSON document which describes what went wrong. A task may terminate with writing an error file which contains a list of such reports. Of course, usually there is only one error, as a typical task implementation will terminate once such a situation is encountered. But HQS Tasks is designed flexible enough to also process multiple errors generated in a task, such as when a validation step finds multiple issues, or when it forwards errors for multiple sub-tasks etc.

Each error report is a JSON object which has the following fields:

  • type (string): The name of the error category, like a primary "classification". For now, we do not define a set of types and the developer can choose these freely. Usually, they correspond to exception class names in the language models. While currently not enforcing it, we recommend to use simple terms written in upper camel case, such as SystemTooLarge.
  • message (string): A message in human-readable form. This is meant to be never machine-read: We explicitly tell developers to not include information only within the message (potentially forcing the client to parse this string). Machine-readable information shall be put in the details section, see below.
  • details (object): A JSON object which collects any additional information, particularly all details relevant for processing this error in a surrounding logic. The object is not further specified here. While also not strictly enforcing it, we recommend developers to keep the structure simple and compatible between versions, and use the same structure for all errors of the same type. The details object generated by our Python implementation includes the exception's arguments and a stacktrace.

Crashes (a.k.a. Abnormal Termination)

When any CLI task exits while not having written a proper output and also not an error report, we assume the task crashed.

The system which processes the response of the task CLI solves this situation by automatically generating an error report which explains this situation to the client.