Failed to play: No storage could be initialized. Abort

[AbhishekTiwari07]

Description

I encountered an issue when attempting to convert a single bag file into multiple compressed bag files. After the conversion, I am unable to play the compressed bag files and encounter an error.

Expected Behavior

I should be able to play the file without any error.

Actual Behavior

I encounter an error while trying to read the 4th bag file.

[ERROR] [1726217343.678629849] [rosbag2_storage]: Could not open 'split_bag/split_bag_3.db3' with 'sqlite3'. Error: Error when processing SQL statement. SQLite error: (11): database disk image is malformed
[ERROR] [1726217343.680619999] [rosbag2_storage]: Could not load/open plugin with storage id 'sqlite3'
[ERROR] [1726217343.743596884] [rosbag2_player]: Failed to play: No storage could be initialized. Abort

To Reproduce

1. Download the attached "reprod_files.zip" and extract "/input_bag" and "output.yaml".
2. Run ros2 bag convert -i <path to input_bag> -o output.yaml
3. Run ros2 bag play split_bag
4. Logs print error while attempting to read 4th bag file (split_bag_3.db3)

System (please complete the following information)

• OS: Ubuntu Noble (24.04)
• ROS 2 Distro: Jazzy
• Install Method: release
• Version: release

Additional context

reprod_files.zip

[AbhishekTiwari07]

I observed this issue in ROS2 Jazzy and found that it also occurs in Iron. However, it works correctly in Humble.

[karthiknit1]

@r7vme @MichaelOrlov - Can you take a look at this, I am also seeing this issue on windows with Jazzy and Iron installation

[r7vme]

@karthiknit1 I was able to reproduce it with Jazzy. Looks like this is smth to do with max_bagfile_size, when compression is enabled. When max_bagfile_size parameter is commented out everything works as expected.

@MichaelOrlov any ideas off the top of your head?

[karthiknit1]

@r7vme In the below function, when I pass empty string instead of message_definition.encoded_message_definition, Its able to convert db3 to zstd and decompress all the bags while playing with no error. I am not sure if the message definition is causing any issue here.

void SqliteStorage::create_topic(
const rosbag2_storage::TopicMetadata & topic,
const rosbag2_storage::MessageDefinition & message_definition)
{
std::lock_guardstd::mutex db_lock(db_read_write_mutex_);
if (topics_.find(topic.name) == std::end(topics_)) {
auto insert_topic =
database_->prepare_statement(
"INSERT INTO topics"
"(name, type, serialization_format, offered_qos_profiles, type_description_hash) "
"VALUES (?, ?, ?, ?, ?)");
insert_topic->bind(
topic.name,
topic.type,
topic.serialization_format,
rosbag2_storage::serialize_rclcpp_qos_vector(topic.offered_qos_profiles),
topic.type_description_hash);
insert_topic->execute_and_reset();
topics_.emplace(topic.name, static_cast(database_->get_last_insert_id()));
}
// TODO(morlov): Add topic.type_description_hash when it will be really calculated or getting
// from service. Currently dummy hashes causing tests failure
std::string topic_type_and_hash = message_definition.topic_type;
if (!topic_type_and_hash.empty() &&
msg_definitions_.find(topic_type_and_hash) == std::end(msg_definitions_))
{
std::string type_description_hash = message_definition.type_hash.empty() ?
topic.type_description_hash : message_definition.type_hash;
auto insert_msg_definition =
database_->prepare_statement(
"INSERT INTO message_definitions (topic_type, encoding, encoded_message_definition, "
"type_description_hash) VALUES (?, ?, ?, ?)");
insert_msg_definition->bind(
message_definition.topic_type, message_definition.encoding,
message_definition.encoded_message_definition, type_description_hash);
insert_msg_definition->execute_and_reset();
msg_definitions_.emplace(
topic_type_and_hash,
static_cast(database_->get_last_insert_id()));
}
db_file_size_ = db_page_size_ * read_total_page_count_locked();
}

[MichaelOrlov]

@AbhishekTiwari07 I am not able to reproduce this issue the way how you did it because don't have message definition for the vizualization_msgs.
Could you please attach what you have got after ros2 bag play split_bag? i.e. content of the split_bag folder.

[MichaelOrlov]

@AbhishekTiwari07 Never mind, I was able to reproduce this issue.
Will need to debug and make a more comprehensive analysis.
It seems the 3rd DB file is really corrupted after conversion.

[MichaelOrlov]

There is something wrong with the compression operation during bag conversion.
Without compression, all bags are not corrupted and successfully replayed.