Skip to content
This repository has been archived by the owner on Nov 13, 2024. It is now read-only.

[Bug]: "failed to query: collection not loaded" #105

Open
mkaramuk opened this issue Sep 23, 2024 · 1 comment
Open

[Bug]: "failed to query: collection not loaded" #105

mkaramuk opened this issue Sep 23, 2024 · 1 comment

Comments

@mkaramuk
Copy link

mkaramuk commented Sep 23, 2024 •
edited
Loading

Current Behavior

I created two new Milvus instances (both of them have version 2.4.0). Then inserted some data to the first one. This is my migration config:

dumper:
  worker:
    workMode: milvus2x # work mode：milvus2x->milvus2x
    reader:
      bufferSize: 500 # Read source data rows in each time read from Source Milvus.

meta: # meta part
  mode: config # 'config' mode means will get meta config from this config file itself.
  version: 2.4.0 #  Source Milvus version
  collection: demo

source: # source milvus connection info
  milvus2x:
    endpoint: <ENDPOINT_1>
    username: root
    password: <PASSWORD>
    grpc:
        maxCallRecvMsgSize: 67108864
        maxCallSendMsgSize: 268435456

target: # target milvus collection info
  milvus2x:
    endpoint: <ENDPOINT_2>
    username: root
    password: <PASSWORD>
    grpc:
        maxCallRecvMsgSize: 67108864
        maxCallSendMsgSize: 268435456

But I am getting error when I try to run migration:

jodId is  starti7W5ywY2F0ENxOmQz8inQ
[2024/09/23 15:50:44.670 +03:00] [INFO] [config/config.go:230] ["input configFile is empty, will read from dir"]
[2024/09/23 15:50:44.670 +03:00] [INFO] [config/resolve_milvus2x.go:88] ["meta.fields param not exists, will migrate all fields"]
[2024/09/23 15:50:44.670 +03:00] [INFO] [dbclient/milvus2x.go:34] ["begin to new milvus2x client"] [endPoint=svc-948-u21416.vm.elestio.app:19530]
[2024/09/23 15:50:44.889 +03:00] [INFO] [dbclient/milvus2x.go:71] ["[Milvus2x] begin to test target connect"] [endpoint=svc-948-u21416.vm.elestio.app:19530] [username=root] [databaseName=] [GrpcMaxCallRecvMsgSize=67108864] [GrpcMaxCallSendMsgSize=268435456]
[2024/09/23 15:50:44.961 +03:00] [INFO] [cmd/start.go:32] ["[Starter] begin to do migration..."]
[2024/09/23 15:50:44.961 +03:00] [INFO] [meta/meta_milvus2x_helper.go:31] ["[MetaMilvus2xHelper] begin to get meta, "] [metaMode=config]
[2024/09/23 15:50:44.961 +03:00] [INFO] [meta/meta_milvus2x_helper.go:37] ["Milvus2x Meta Info"] [version=2.4.11]
[2024/09/23 15:50:44.961 +03:00] [INFO] [meta/meta_milvus2x_helper.go:39] ["[Milvus2x Meta Static] Collection"] [collection=demo]
[2024/09/23 15:50:44.961 +03:00] [INFO] [migration/milvus2x_starter.go:16] ["dump Milvus2x split collections for concurrent work"] [CollectionSize=1] [ConcurLimit=1]
[2024/09/23 15:50:44.961 +03:00] [INFO] [milvus2x_factory/milvus2x_factory.go:39] ["[Milvus2x Factory] begin to new milvus2x client"] [endpoint=svc-949-u21416.vm.elestio.app:19530] [version=2.4.11]
[2024/09/23 15:50:44.961 +03:00] [WARN] [milvus2x/base.go:44] ["milvus2x version not contain, will use default sdk version"] [Version=2.4.11]
[2024/09/23 15:50:44.961 +03:00] [INFO] [milvus2x/milvus2_3_ver.go:134] ["[Milvus23x] begin to new milvus client"] [endPoint=svc-949-u21416.vm.elestio.app:19530]
[2024/09/23 15:50:45.176 +03:00] [INFO] [milvus2x/milvus2_3_ver.go:170] ["[Milvus23x] begin to test source connect"] [endpoint=svc-949-u21416.vm.elestio.app:19530] [username=root] [databaseName=] [GrpcMaxCallRecvMsgSize=67108864] [GrpcMaxCallSendMsgSize=268435456]
[2024/09/23 15:50:45.386 +03:00] [INFO] [convert/milvus2x_convert.go:36] ["milvus2x source collection_schema"] [DynamicFieldStatus=false] [Collection=demo] [PartitionKey=] [Partitions="[{\"ID\":452747360140264041,\"Name\":\"_default\",\"Loaded\":false}]"]
[2024/09/23 15:50:45.387 +03:00] [INFO] [convert/milvus2x_convert.go:110] ["milvus2x transform to fillAllFields Milvus"] ["srcField AutoId"=false]
[2024/09/23 15:50:45.387 +03:00] [INFO] [convert/milvus2x_convert.go:156] ["milvus2x transform custom target Milvus"] [AutoId=]
[2024/09/23 15:50:45.387 +03:00] [INFO] [loader/cus_milvus2x_loader.go:45] ["[Loader] All collection Begin to create..."]
[2024/09/23 15:50:45.457 +03:00] [WARN] [dbclient/cus_field_milvus2x.go:45] ["collection already exist,no need to create"] [collectionName=demo]
[2024/09/23 15:50:45.457 +03:00] [INFO] [loader/cus_milvus2x_loader.go:45] ["[Loader] Create All collection finish."]
[2024/09/23 15:50:45.457 +03:00] [INFO] [loader/cus_milvus2x_loader.go:49] ["[Loader] collection load data before statistics..."]
[2024/09/23 15:50:45.457 +03:00] [INFO] [dbclient/cus_field_milvus2x.go:107] ["[Milvus2x] begin to ShowCollectionRows"]
[2024/09/23 15:50:45.606 +03:00] [INFO] [dbclient/cus_field_milvus2x.go:107] ["[Milvus2x] Target Collection Static:"] [collection=demo] [rowCount=0]
[2024/09/23 15:50:45.679 +03:00] [ERROR] [migration/milvus2x_starter.go:53] ["DumpByIterator err"] [error="failed to query: collection not loaded[collection=452747360140264040]"] [errorVerbose="failed to query: collection not loaded[collection=452747360140264040]\n(1) attached stack trace\n  -- stack trace:\n  | github.com/milvus-io/milvus-sdk-go/v2/client.handleRespStatus\n  | \t/home/mkaramuk/go/pkg/mod/github.com/milvus-io/milvus-sdk-go/[email protected]/client/collection.go:39\n  | github.com/milvus-io/milvus-sdk-go/v2/client.(*GrpcClient).Query\n  | \t/home/mkaramuk/go/pkg/mod/github.com/milvus-io/milvus-sdk-go/[email protected]/client/data.go:349\n  | github.com/zilliztech/milvus-migration/storage/milvus2x.(*Milvus23VerClient).Count\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/storage/milvus2x/milvus2_3_ver.go:96\n  | github.com/zilliztech/milvus-migration/core/reader/source.(*Milvus2xSource).Count\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/core/reader/source/milvus2x_source.go:100\n  | github.com/zilliztech/milvus-migration/core/dumper.(*Dumper).ReadData2Channel\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/core/dumper/mode_starter_milvus2x.go:47\n  | github.com/zilliztech/milvus-migration/core/dumper.(*Dumper).WorkInMilvus2x\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/core/dumper/mode_starter_milvus2x.go:34\n  | github.com/zilliztech/milvus-migration/starter/migration.(*Starter).dumpByIterator\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/starter/migration/milvus2x_starter.go:66\n  | github.com/zilliztech/milvus-migration/starter/migration.(*Starter).DumpLoadInMilvus2x.func2\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/starter/migration/milvus2x_starter.go:50\n  | golang.org/x/sync/errgroup.(*Group).Go.func1\n  | \t/home/mkaramuk/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78\n  | runtime.goexit\n  | \t/usr/local/go/src/runtime/asm_amd64.s:1695\nWraps: (2) failed to query: collection not loaded[collection=452747360140264040]\nError types: (1) *withstack.withStack (2) *errutil.leafError"] [stack="github.com/zilliztech/milvus-migration/starter/migration.(*Starter).DumpLoadInMilvus2x.func2\n\t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/starter/migration/milvus2x_starter.go:53\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/mkaramuk/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78"]
[2024/09/23 15:50:45.679 +03:00] [ERROR] [cmd/start.go:34] ["[start migration error]"] [error="failed to query: collection not loaded[collection=452747360140264040]"] [errorVerbose="failed to query: collection not loaded[collection=452747360140264040]\n(1) attached stack trace\n  -- stack trace:\n  | github.com/milvus-io/milvus-sdk-go/v2/client.handleRespStatus\n  | \t/home/mkaramuk/go/pkg/mod/github.com/milvus-io/milvus-sdk-go/[email protected]/client/collection.go:39\n  | github.com/milvus-io/milvus-sdk-go/v2/client.(*GrpcClient).Query\n  | \t/home/mkaramuk/go/pkg/mod/github.com/milvus-io/milvus-sdk-go/[email protected]/client/data.go:349\n  | github.com/zilliztech/milvus-migration/storage/milvus2x.(*Milvus23VerClient).Count\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/storage/milvus2x/milvus2_3_ver.go:96\n  | github.com/zilliztech/milvus-migration/core/reader/source.(*Milvus2xSource).Count\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/core/reader/source/milvus2x_source.go:100\n  | github.com/zilliztech/milvus-migration/core/dumper.(*Dumper).ReadData2Channel\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/core/dumper/mode_starter_milvus2x.go:47\n  | github.com/zilliztech/milvus-migration/core/dumper.(*Dumper).WorkInMilvus2x\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/core/dumper/mode_starter_milvus2x.go:34\n  | github.com/zilliztech/milvus-migration/starter/migration.(*Starter).dumpByIterator\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/starter/migration/milvus2x_starter.go:66\n  | github.com/zilliztech/milvus-migration/starter/migration.(*Starter).DumpLoadInMilvus2x.func2\n  | \t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/starter/migration/milvus2x_starter.go:50\n  | golang.org/x/sync/errgroup.(*Group).Go.func1\n  | \t/home/mkaramuk/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78\n  | runtime.goexit\n  | \t/usr/local/go/src/runtime/asm_amd64.s:1695\nWraps: (2) failed to query: collection not loaded[collection=452747360140264040]\nError types: (1) *withstack.withStack (2) *errutil.leafError"] [stack="github.com/zilliztech/milvus-migration/cmd.init.func6\n\t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/cmd/start.go:34\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/mkaramuk/go/pkg/mod/github.com/spf13/[email protected]/command.go:987\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/mkaramuk/go/pkg/mod/github.com/spf13/[email protected]/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/mkaramuk/go/pkg/mod/github.com/spf13/[email protected]/command.go:1039\ngithub.com/zilliztech/milvus-migration/cmd.Execute\n\t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/cmd/root.go:27\nmain.main\n\t/home/mkaramuk/Desktop/projects/playground/milvus/milvus-migration/main.go:16\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:271"]

Expected Behavior

I was expecting to perform migration successfully.

Steps To Reproduce

1. Create two different Milvus deployment
2. Use the script below to fill one of them with data:

import random

from pymilvus import (
    connections,
    FieldSchema, CollectionSchema, DataType,
    Collection,
    utility
)

# This example shows how to:
#   1. connect to Milvus server
#   2. create a collection
#   3. insert entities
#   4. create index
#   5. search


_HOST = 'host'
_PORT = '19530'

# Const names
_COLLECTION_NAME = 'demo'
_ID_FIELD_NAME = 'id_field'
_VECTOR_FIELD_NAME = 'float_vector_field'

# Vector parameters
_DIM = 128
_INDEX_FILE_SIZE = 32  # max file size of stored index

# Index parameters
_METRIC_TYPE = 'L2'
_INDEX_TYPE = 'IVF_FLAT'
_NLIST = 1024
_NPROBE = 16
_TOPK = 3


# Create a Milvus connection
def create_connection():
    print(f"\nCreate connection...")
    connections.connect(host=_HOST, port=_PORT, user="root", password="passwd")
    print(f"\nList connections:")
    print(connections.list_connections())


# Create a collection named 'demo'
def create_collection(name, id_field, vector_field):
    field1 = FieldSchema(name=id_field, dtype=DataType.INT64, description="int64", is_primary=True)
    field2 = FieldSchema(name=vector_field, dtype=DataType.FLOAT_VECTOR, description="float vector", dim=_DIM,
                         is_primary=False)
    schema = CollectionSchema(fields=[field1, field2], description="collection description")
    collection = Collection(name=name, data=None, schema=schema, properties={"collection.ttl.seconds": 15})
    print("\ncollection created:", name)
    return collection


def has_collection(name):
    return utility.has_collection(name)


# Drop a collection in Milvus
def drop_collection(name):
    collection = Collection(name)
    collection.drop()
    print("\nDrop collection: {}".format(name))


# List all collections in Milvus
def list_collections():
    print("\nlist collections:")
    print(utility.list_collections())


def insert(collection, num, dim):
    data = [
        [i for i in range(num)],
        [[random.random() for _ in range(dim)] for _ in range(num)],
    ]
    collection.insert(data)
    return data[1]


def get_entity_num(collection):
    print("\nThe number of entity:")
    print(collection.num_entities)


def create_index(collection, filed_name):
    index_param = {
        "index_type": _INDEX_TYPE,
        "params": {"nlist": _NLIST},
        "metric_type": _METRIC_TYPE}
    collection.create_index(filed_name, index_param)
    print("\nCreated index:\n{}".format(collection.index().params))


def drop_index(collection):
    collection.drop_index()
    print("\nDrop index sucessfully")


def load_collection(collection):
    collection.load()


def release_collection(collection):
    collection.release()


def search(collection, vector_field, id_field, search_vectors):
    search_param = {
        "data": search_vectors,
        "anns_field": vector_field,
        "param": {"metric_type": _METRIC_TYPE, "params": {"nprobe": _NPROBE}},
        "limit": _TOPK,
        "expr": "id_field >= 0"}
    results = collection.search(**search_param)
    for i, result in enumerate(results):
        print("\nSearch result for {}th vector: ".format(i))
        for j, res in enumerate(result):
            print("Top {}: {}".format(j, res))


def set_properties(collection):
    collection.set_properties(properties={"collection.ttl.seconds": 1800})


def main():
    # create a connection
    create_connection()

    # drop collection if the collection exists
    if has_collection(_COLLECTION_NAME):
        drop_collection(_COLLECTION_NAME)

    # create collection
    collection = create_collection(_COLLECTION_NAME, _ID_FIELD_NAME, _VECTOR_FIELD_NAME)
    collection2 = create_collection(_COLLECTION_NAME + "2", _ID_FIELD_NAME + "2", _VECTOR_FIELD_NAME+ "2")

    # alter ttl properties of collection level
    set_properties(collection)

    # show collections
    list_collections()

    # insert 10000 vectors with 128 dimension
    vectors = insert(collection, 10000, _DIM)

    # create index
    create_index(collection, _VECTOR_FIELD_NAME)

    collection.flush()

"""     # load data to memory
    load_collection(collection)

    # search
    search(collection, _VECTOR_FIELD_NAME, _ID_FIELD_NAME, vectors[:3])

    # release memory
    release_collection(collection)
    collection.flush() """
""" 
    # drop collection index
    drop_index(collection)

    # drop collection
    drop_collection(_COLLECTION_NAME)
 """

if __name__ == '__main__':
    main()


> You might need to install `pymilvus` package

3. Create config file like the one that I've created (it is in the issue message) and put it into correct directory (configs/migration.yaml)
4. Run `./milvus-migration start` command

Environment

Nothing special

Anything else?

No response
@xiaofan-luan
Copy link
Collaborator

I think the source cluster colleection is not loaded, collectionID 452747360140264040.

BTW, we highly recommend to use 2.4.12 instead of 2.4.0

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mkaramuk @xiaofan-luan