EXTENSIONS.md

Adding new keywords

Adding kew keywords to the program should be rather easy. The general APRS message work flow is as follows:

Part 1 - Receiving the input string

If an APRS messages has been received and has made it past the primary input filter (see INSTALLATION), MPAD examines the message and extracts certain information from that message:

• who has sent me the message (from_callsign)
• the adresse callsign that is supposed to receive the message. This is the message recipient - usually, it's our own bot process' call sign.
• the actual message text
• message number if present in the message
• and finally, we extract the format string of that APRS message.

In order to enter the input parser, the following preconditions need to be met:

• format has to be message. If the value is response, we ignore that APRS message.
• from_callsign has to be in the program's second filter. This is not the APRS-IS server filter but the filter that is rum by MPAD itself. See the installation instructions on how to configure this setting.
• The incoming message must not be a duplicate. For each message that is sent to MPAD, a hash key is built and added to a decaying cache. If the same message is received within 5 mins, it will be detected as a duplicate request. The message will neither re-acked nor processed. If your ham radio uses message IDs for APRS messages, sending the same message text content will not trigger a duplicate detection if the message numbers differ. See the TECHNICAL_DETAILS on how the dupe check works.

Finally, MPAD has a look for some corrupted APRS messages where the message ID is present but not properly transmitted. aprslib does not detect these message IDs (since the message itself does not follow APRS standards). MPAD tries to extract and repair that data, if necessary.

If that previous process has found a message ID (or it has been previously identified), we will first send an acknowledgment to APRS-IS and then continue with parsing the data.

Part 2 - Parsing the data

This is the part where we need to get our hands dirty. Herem we try to figure out what the user wants us to do. We do not perform those actual tasks (e.g. pull the wx report) in here - these are parts of the the output generator (see next chapter)

input_parser.py has three input parameters:

• the aprs message text
• the from_callsign ("who has sent me the data")
• and finally, the API key to aprs.fi

Output of the function consists of two variables:

• a general success variable. If that one is set to True, we did not encounter any errors. If the value is False, the output generator will send an error message to the user
• a dictionary which contains keyword-specific values. Dependent on what you asked the program to do, these values may or may not be populated.

That data structure comes with a couple of fields which are of universal nature. The most important ones:

• what contains the actual command that the user wants us to execute (e.g. 'get wx data', 'get position for user xyz'). The output parser will extract this field and then calls the respective subroutines for generating the outgoing content.
• when countains the normalized reference to a day, e.g. 'monday'. date_offset already represents its numerical nature in relation to the server's date settings. For example, if the user has requested data for 'Thursday' on a Tuesday, that field's value is '2'.
• when_daytime contains a normalized reference to a certain daytime (e.g. 'night', 'evening' etc)
• latitude and longitude. Self-explanatory. These fields always contain the user's coordinates or -if a different call sign is to be used as position reference- that user's call sign. Ann keywords use these lat/lon coordinates as their queries' point of destination, meaning that additional information such as city name, country etc is only used for data output to the user. The additional fields users_latitude and users_longitude are only used for the whereis command - for any other keyword, they are not populated.
• units represents the units of measure that the program will use (metric (default) or imperial)

Parsing the data in a nutshell:

Based on keyword-specific regex commands, the existing message text will be parsed. If a regex query was successful, MPAD will extract the information (related to the associated regex) and then set the what keyword with the command that the user has requested. Afterwards, we perform a little bit of a cleanup:

• The regex'ed string will be removed from the originating APRS message. As we need to continue our parser process (so far, we've only figured out 'what' the user wants us to do; the 'when' question is still unresolved), keeping that data might cause some unwanted effects.
• Finally, a general marker called found_my_duty_roster will be set. If that marker is set to True, the parser knows that it has found a 'what' command that it is required to execute at a later point in time.

The parser starts with a query for multiple keywords such as whereis, followed by CWOP/Position/Celestial keywords. The first successful regex 'wins', meaning that if you e.g. create a message where you query for METAR data and satellite data, the METAR data keyword wins as this is one of the first regex keyword queries in the program.

At the end of the parsing process where we have taken a look at the complete message, the following things happen:

• The remaining APRS message text will be split up (separator = blank). For each of these string words, another parser round will be issued. This is also the time where the when and when_daytime values are determined. If no 'real' when keyword has been found, the program will try to provide you a wx report for the current user's position. This is the prgram's default fallback
• All internal values will be added to that output dictionary response_parameters. It will then be the output generator's responsibility to turn these requests into something useful.

Part 3 - Generating the output string

output_generator.py is responsible for generating the outgoing messages to the user. Dependent on the what keyword, the program calls the respective functions which will then do the actual work (e.g. get wx report etc.). Each of these functions uses a function called make_pretty_aprs_messages in order to add the content to a List item which contains 1..n lines of ready-to-be-sent text messages. Each of these messages has a max len of 67 characters, so you won't exceed the max. APRS message length. make_pretty_aprs_messages is responsible for a couple of things:

• optional removal of non-ASCII characters from the content if the program configuration requests this (see INSTALLATION)
• Initially, call make_pretty_aprs_messages without a List reference and provide your string. As a result, you'll get a reference to the output List item which contains your string.

For each new string that you are going to add, make_pretty_aprs_messages checks if the len for (current existing string plus your new string) exceed 67 characters. If that is the case, a new element is genarated (which represents your current input to the function). make_pretty_aprs_messages always tries to add the content without ripping it apart (e.g. the 67th byte of a message contains "1" and the 1st byte contains "2" for a temperature reference of 12 degrees. Yes, this might result in 'bloated' APRS messages so ensure that you add your content in a proper manner (see Testing).

There are a few safety nets: for the unlikely event of receiving an input string of more than 67 characters, MPAD refrains from keeping the logical connection and tries to split up that string on a per-word basis. This does not rip content apart but may break the logical connection. Finally, if there should ever be a string of more than 67 characters without any blanks (Martian landing coordinates?), MPAD will break up that string into chunks of 67 bytes. MPAD does not speak Martian and neither do I. This is the 'last resort' option which should never be triggered.

Here's how to call make_pretty_aprs_messages:

• Start with the process by calling make_pretty_aprs_messages without a List item reference. You'll get a reference to the List object which contains your first message.
• For message contents 2..n, pass that reference to the function in order to ensure that the next message is added to that same List.
• For each new message, that you add, a separator (default: space) is added between the previous message and the new message. If you don't want that message separator to be added, you can omit it by specifying the add_sep parameter.

Once your native routine has prepared that list, there is nothing else that you need to do. If the original incoming message was sent with a message ID, MPAD will automatically add unique message IDs to each outgoing message. Finally, the(se) message(s) are sent to APRS-IS.

The output generator has only a single parameter:

• the dictionary from the input_parser.py module

Entering the output parser ONLY happens if the input parser has found a valid what command. Otherwise, a generic error message is presented to the user.

Test your new keyword

For a non-live test of the input parser and output generator, you can use the parser_test.py Python file which is part of this repo. Simply specify your own call sign (the one that you would send the message from) and the APRS message. Both input_parser.py and output_generator.py will be triggered, and you can test if your new keyword works as designed. This is also good for testing if all Python packages have been installed and everything has been configured in a proper way.