Skip to content

Commit

Permalink
Multi-station support (#9)
Browse files Browse the repository at this point in the history
Supports recording multiple stations
  • Loading branch information
rmens authored Dec 19, 2024
1 parent a9bcd1d commit 673ab9e
Show file tree
Hide file tree
Showing 3 changed files with 187 additions and 83 deletions.
89 changes: 66 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,23 @@
# Audiologger ZuidWest FM

This repository contains a bash script designed to record audio streams hourly and log relevant metadata about the current broadcast. It also ensures the periodic cleanup of old recordings.

## Features

- **Continuous Recording**: Automatically captures the audio stream from ZuidWest FM every hour.
- **Metadata Logging**: Fetches and logs the current program name from the broadcast data API, adding context to each recording.
- **Detailed Log File**: Maintains a comprehensive log file for tracking the script’s activities and any potential errors.
- **Automatic Cleanup**: Deletes audio files older than 31 days to conserve storage space.
- **Continuous Recording**: Automatically captures audio streams every hour.
- **Metadata Logging**: Fetches and logs the current program name from broadcast data APIs, adding context to each recording.
- **Detailed Log File**: Maintains a comprehensive log file for tracking the script's activities and any potential errors.
- **Automatic Cleanup**: Deletes audio files based on configurable retention periods.
- **Debug Mode**: Provides additional output for troubleshooting when enabled.
- **Multi-Stream Support**: Can record multiple streams simultaneously with different configurations.

## Prerequisites

The script requires the following tools:
- `jq` - A command-line JSON processor.
- `curl` - A command-line tool for transferring data with URLs.
- `ffmpeg` - A command-line tool for recording, converting, and streaming audio and video.

This script is intended for use with websites based on the [Streekomroep WordPress Theme](https://github.com/oszuidwest/streekomroep-wp), which utilizes the Broadcast Data API from the theme. If you are using a different API, set `PARSE_METADATA` to 0 and use a plaintext file for metadata, or implement your own parsing method.
This script is intended for use with websites based on the [Streekomroep WordPress Theme](https://github.com/oszuidwest/streekomroep-wp), which utilizes the Broadcast Data API from the theme. If you are using a different API, set `parse_metadata` to 0 and use a plaintext file for metadata.

## Installation

1. **Clone this repository:**
```bash
git clone https://github.com/oszuidwest/zwfm-audiologger
Expand All @@ -32,17 +29,66 @@ This script is intended for use with websites based on the [Streekomroep WordPre
```

## Configuration

Edit the script to specify the recording directory (`RECDIR`), log file path (`LOGFILE`), and other parameters:
- `STREAMURL`: The URL of the audio stream.
- `RECDIR`: The directory where audio recordings are stored.
- `LOGFILE`: The path to the log file for logging script operations.
- `METADATA_URL`: The API endpoint for fetching broadcast metadata.
- `KEEP`: The number of days to retain audio recordings.
- `PARSE_METADATA`: Enables or disables metadata parsing from the Streekomroep WordPress theme.
Configuration is done through `streams.json`. The file has two main sections: `global` and `streams`.

### Global Settings
```json
{
"global": {
"rec_dir": "/var/audio", // Where to store recordings
"log_file": "/var/log/audiologger.log", // Log file location
"keep_days": 31, // Default retention period
"debug": 1 // Enable console logging
}
}
```

All global settings can be customized.

### Stream Settings
Each stream in the `streams` section can have these settings:

```json
{
"streams": {
"stream_name": { // Name used for subdirectory
"stream_url": "https://...", // Stream URL
"metadata_url": "https://...", // Metadata URL
"metadata_path": ".some.path", // JSON path for metadata (if parsing)
"parse_metadata": 1, // Parse JSON (1) or use raw response (0)
"keep_days": 31 // Override global keep_days
}
}
}
```

#### Customizable per stream:
- `stream_url`: The URL of the audio stream
- `metadata_url`: Where to fetch program information
- `metadata_path`: JSON path for metadata extraction (only if parse_metadata: 1)
- `parse_metadata`: Whether to parse JSON response (1) or use raw response (0)
- `keep_days`: How long to keep recordings

#### Fixed settings (do not override):
- Recording time is fixed at 1 hour (3600 seconds)
- Network settings:
- `reconnect_delay_max`: 300 seconds
- `rw_timeout`: 10000000
- Error codes: 404, 500, 503

### Directory Structure
The script creates:
```
/var/audio/
├── stream_name1/
│ ├── 2024-12-19_14.mp3
│ └── 2024-12-19_14.meta
└── stream_name2/
├── 2024-12-19_14.mp3
└── 2024-12-19_14.meta
```

## Usage

Schedule the script to run every hour using cron:
1. Open your crontab:
```bash
Expand All @@ -54,15 +100,12 @@ Schedule the script to run every hour using cron:
```

## Debugging

To enable debug mode, set `DEBUG=1` in the script. This will output debug information to the console to help identify any issues during execution.
To enable debug mode, set `debug: 1` in the global section of streams.json. This will output debug information to the console to help identify any issues during execution.

## Contributing

Contributions are welcome. Please fork the repository, make your changes, and submit a pull request.

# MIT License

Copyright (c) 2024 Streekomroep ZuidWest

Permission is hereby granted, free of charge, to any person obtaining a copy
Expand All @@ -81,4 +124,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
157 changes: 97 additions & 60 deletions audiologger.sh
Original file line number Diff line number Diff line change
@@ -1,75 +1,112 @@
#!/bin/bash
# Records hourly segments from a live stream and stores program metadata
# Designed to be run via cron every hour

# Configuration
STREAMURL='https://icecast.zuidwestfm.nl/zuidwest.mp3'
RECDIR='/var/audio'
LOGFILE='/var/log/audiologger.log'
METADATA_URL='https://www.zuidwestupdate.nl/wp-json/zw/v1/broadcast_data'
# Output date and hour, e.g., "2023_12_31_20"
TIMESTAMP=$(/bin/date +"%Y-%m-%d_%H")
# Number of days to keep the audio files
KEEP=31
# Debug mode flag (set to 1 to enable debug mode)
DEBUG=1
# Metadata parsing flag (set to 1 to enable metadata parsing, 0 for plain text)
PARSE_METADATA=1
CONFIG_FILE=${CONFIG_FILE:-'/app/streams.json'}

# Function to handle logging
log_message() {
local message="$1"
echo "$(date): $message" >> "$LOGFILE"
if [ "$DEBUG" -eq 1 ]; then
echo "$(date): $message"
fi
# Check if config exists
if [[ ! -f "$CONFIG_FILE" ]]; then
echo "ERROR: Config file not found: $CONFIG_FILE"
exit 1
fi

# Load global settings
RECDIR=$(jq -r '.global.rec_dir' "$CONFIG_FILE")
LOGFILE=$(jq -r '.global.log_file' "$CONFIG_FILE")
DEBUG=$(jq -r '.global.debug' "$CONFIG_FILE")
KEEP=$(jq -r '.global.keep_days' "$CONFIG_FILE")

# Setup logging
mkdir -p "$(dirname "$LOGFILE")"
touch "$LOGFILE"

# Log with timestamps
log() {
echo "$(date '+%Y-%m-%d %H:%M:%S'): $1" >> "$LOGFILE"
[[ $DEBUG -eq 1 ]] && echo "$(date '+%Y-%m-%d %H:%M:%S'): $1"
}

# Ensure the log file exists
if [ ! -f "$LOGFILE" ]; then
mkdir -p "$(dirname "$LOGFILE")"
touch "$LOGFILE"
fi
# Function to fetch and store metadata
fetch_metadata() {
local name=$1
local metadata_url=$2
local metadata_path=$3
local parse_metadata=$4
local stream_dir=$5
local timestamp=$6

# Get program info
{
if [[ $parse_metadata -eq 1 && -n "$metadata_path" ]]; then
PROGRAM_NAME=$(curl -s --max-time 15 "$metadata_url" 2>/dev/null | jq -r "$metadata_path")
else
PROGRAM_NAME=$(curl -s --max-time 15 "$metadata_url" 2>/dev/null)
fi

[[ -z "$PROGRAM_NAME" || "$PROGRAM_NAME" == "null" ]] && PROGRAM_NAME="Unknown Program"

# Store metadata
echo "$PROGRAM_NAME" > "${stream_dir}/${timestamp}.meta" || \
log "WARNING: Failed to write metadata for $name"

log "INFO: Stored metadata for $name - $timestamp - $PROGRAM_NAME"
} &
}

# Check if required commands are installed (ffmpeg, curl, jq)
# Check dependencies
for cmd in ffmpeg curl jq; do
if ! command -v $cmd &> /dev/null; then
log_message "$cmd is not installed."
log "ERROR: $cmd is not installed."
exit 1
fi
done

# Create recording directory if it does not exist
if [ ! -d "$RECDIR" ]; then
mkdir -p "$RECDIR" || { log_message "Failed to create directory: $RECDIR"; exit 1; }
fi

# Remove old files based on the KEEP variable
find "$RECDIR" -type f -mtime "+$KEEP" -exec rm {} \; || log_message "Failed to remove old files in $RECDIR"

# Check if an ffmpeg process with the specified stream URL and timestamp is already running
if pgrep -af "ffmpeg.*$STREAMURL.*$TIMESTAMP" > /dev/null; then
log_message "An ffmpeg recording process for $TIMESTAMP with $STREAMURL is already running. Exiting."
exit 1
fi
# Create base directory
mkdir -p "$RECDIR"
TIMESTAMP=$(date +"%Y-%m-%d_%H")

# Fetch current program name
if [ "$PARSE_METADATA" -eq 1 ]; then
# Parse metadata using jq
PROGRAM_NAME=$(curl --silent "$METADATA_URL" | jq -r '.fm.now')
if [ -z "$PROGRAM_NAME" ] || [ "$PROGRAM_NAME" == "null" ]; then
log_message "Failed to fetch current program name or program name is null"
PROGRAM_NAME="Unknown Program"
fi
else
# Use plain value of what the URL displays
PROGRAM_NAME=$(curl --silent "$METADATA_URL")
if [ -z "$PROGRAM_NAME" ]; then
log_message "Failed to fetch current program name"
PROGRAM_NAME="Unknown Program"
# Process each stream
while read -r stream_base64; do
# Decode stream config
stream_json=$(echo "$stream_base64" | base64 -d)
name=$(echo "$stream_json" | jq -r '.key')
stream_url=$(echo "$stream_json" | jq -r '.value.stream_url')
metadata_url=$(echo "$stream_json" | jq -r '.value.metadata_url')
metadata_path=$(echo "$stream_json" | jq -r '.value.metadata_path // empty')
parse_metadata=$(echo "$stream_json" | jq -r '.value.parse_metadata // 0')
keep_days=$(echo "$stream_json" | jq -r '.value.keep_days // empty')

# Use stream-specific keep_days or fall back to global KEEP
[[ -z "$keep_days" ]] && keep_days=$KEEP

# Create and clean stream directory
stream_dir="$RECDIR/$name"
mkdir -p "$stream_dir"
find "$stream_dir" -type f -mtime "+$keep_days" -exec rm {} \; 2>/dev/null || \
log "WARNING: Failed to cleanup old files for $name"

# Check for existing recording
if pgrep -af "ffmpeg.*$stream_url.*$TIMESTAMP" > /dev/null; then
log "WARNING: Recording for $name $TIMESTAMP already running"
continue
fi
fi

# Write metadata to a file
echo "$PROGRAM_NAME" > "${RECDIR}/${TIMESTAMP}.meta" || { log_message "Failed to write metadata file"; exit 1; }

# Start metadata fetch in background
fetch_metadata "$name" "$metadata_url" "$metadata_path" "$parse_metadata" "$stream_dir" "$TIMESTAMP"

# Start recording
log "INFO: Starting recording for $name - $TIMESTAMP"
ffmpeg -nostdin -loglevel error \
-t 3600 \
-reconnect 1 \
-reconnect_at_eof 1 \
-reconnect_streamed 1 \
-reconnect_delay_max 300 \
-reconnect_on_http_error 404,500,503 \
-rw_timeout 10000000 \
-i "$stream_url" \
-c copy \
-f mp3 \
-y "${stream_dir}/${TIMESTAMP}.mp3" 2>> "$LOGFILE" & disown

# Record next hour's stream
ffmpeg -loglevel error -t 3600 -reconnect 1 -reconnect_at_eof 1 -reconnect_streamed 1 -reconnect_delay_max 300 -reconnect_on_http_error 404 -rw_timeout 10000000 -i "$STREAMURL" -c copy -f mp3 -y "${RECDIR}/${TIMESTAMP}.mp3" & disown
done < <(jq -r '.streams | to_entries[] | @base64' "$CONFIG_FILE")
24 changes: 24 additions & 0 deletions streams.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"global": {
"rec_dir": "/var/audio",
"log_file": "/var/log/audiologger.log",
"keep_days": 31,
"debug": 1
},
"streams": {
"zuidwest": {
"stream_url": "https://icecast.zuidwest.cloud/zuidwest.mp3",
"metadata_url": "https://www.zuidwestupdate.nl/wp-json/zw/v1/broadcast_data",
"metadata_path": ".fm.now",
"parse_metadata": 1,
"keep_days": 31
},
"rucphen": {
"stream_url": "https://icecast.zuidwest.cloud/radiorucphen.mp3",
"metadata_url": "https://rucphen.zuidwest.cloud/proxy.php",
"metadata_path": ".fm.now",
"parse_metadata": 1,
"keep_days": 31
}
}
}

0 comments on commit 673ab9e

Please sign in to comment.