Where is data stored

[davidlis]

Where is data for the inserted embeddings stored with the standalone version of Milvus?
(i understand that this is handled by minio but don't see where this lands by default)

Looking at the default docker-compose, I thought it would be in the anonymous volumes created along with the containers. But sudo docker system df says that the volumes have a size of 0B even after inserting lots of vectors into the DB.

Thank you

[yhmo]

The standalone docker-compose file is:
wget https://github.com/milvus-io/milvus/releases/download/v2.1.4/milvus-standalone-docker-compose.yml -O docker-compose.yml

In this yml file, you can see the configuration of minio:

minio:
    container_name: milvus-minio
    image: minio/minio:RELEASE.2022-03-17T06-34-49Z
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data

The "volumes" defines the storage path. After you start the standalone server by docker-compose up -d, you will see a folder "volumes" created under the path where you run the command. All the data files are stored in the ./volumes/minio

[baiwfg2]

I see this statement from https://milvus.io/docs/product_faq.md

Where does Milvus store data?
Milvus deals with two types of data, inserted data and metadata.

Inserted data, including vector data, scalar data, and collection-specific schema, are stored in persistent storage as incremental log. Milvus supports multiple object storage backends, including [MinIO](https://min.io/), [AWS S3](https://aws.amazon.com/s3/?nc1=h_ls), [Google Cloud Storage](https://cloud.google.com/storage?hl=en#object-storage-for-companies-of-all-sizes) (GCS), [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs), [Alibaba Cloud OSS](https://www.alibabacloud.com/product/object-storage-service), and [Tencent Cloud Object Storage](https://www.tencentcloud.com/products/cos) (COS).

That is to say, milvus doesn't have its own storage engine, it just use external storage service to store all data except metadata ? @yhmo

[hxlsky]

metadata is store at ETCD, queue message data store at kafka or Pulsar ,milvus has no own storeage engine.

[yhmo]

Each milvus cluster has three third-party services:

• Pulsar/Kafka acts as a message queue to store WAL(write-ahead-log) for insert/delete requests, and time-tick messages for data consistency. All the inserted data will be stored in Pulsar/Kafka before being consumed by datanode/querynode.
• MinIO/S3 is the major data storage of Milvus, all the original data and index data are stored here. The data is organized as collection/partition/segment/field. Data is stored as column-based.
• Etcd stores the system metadata of Milvus, including schema of collections, runtime address of each component, paths of data/index of each segment, etc.

[xiaofan-luan]

The design goal for milvus it to become stateless, so we can scale fast with the help of kubernetes