Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEATURE: Add a mop upsert command #292

Merged
merged 1 commit into from
Jul 22, 2024
Merged

FEATURE: Add a mop upsert command #292

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions docs/06-map-API.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Map item에 대해 수행 가능한 기본 연산들은 아래와 같다.

- [Map Item 생성](06-map-API.md#map-item-생성) (Map Item 삭제는 key-value item 삭제 함수로 수행한다)
- [Map Element 삽입](06-map-API.md#map-element-삽입)
- [Map Element Upsert](06-map-API.md#map-element-upsert)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

여기만 영문 사용하는 것이 일관성이 없습니다.
java client 쪽과 논의하여 맞추도록 하세요.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

java client도 bop의 upsert 만 영문으로 사용되고 있습니다. 영어단어를 그대로 사용해도 괜찮을까요?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bop upsert가 그렇다면, mop upsert도 동일하게 합시다.

- [Map Element 변경](06-map-API.md#map-element-변경)
- [Map Element 삭제](06-map-API.md#map-element-삭제)
- [Map Element 조회](06-map-API.md#map-element-조회)
Expand Down Expand Up @@ -179,6 +180,71 @@ void arcus_map_element_insert(memcached_st *memc)
}
```

## Map Element Upsert

Map에 하나의 element를 upsert한다.
Upsert 연산은 해당 element가 없으면 insert하고, 있으면 update하는 연산이다.

``` c

memcached_return_t
memcached_mop_upsert(memcached_st *ptr,
const char *key, size_t key_length,
const char *mkey, size_t mkey_length,
const char *value, size_t value_length,
memcached_coll_create_attrs_st *attributes)
```

- key, key_length: map item의 key
- mkey, mkey_length: upsert할 element의 mkey
- value, value_lenth: upsert할 element의 value
- attributes: Map 없을 시에 attributes에 따라 empty map을 생성 후에 element 삽입한다.

Response code는 아래와 같다.

- MEMCACHED_SUCCESS
- MEMCACHED_STORED: 기존에 존재하던 Map에 element가 삽입됨.
- MEMCACHED_CREATED_STORED: Map가 새롭게 생성되고 element가 삽입됨.
- MEMCACHED_REPLACED: 기존에 존재하던 Map에서 동일한 mkey를 가진 element를 대체함.
- not MEMCACHED_SUCCESS
- MEMCACHED_NOTFOUND: Map이 존재하지 않음.
- MEMCACHED_TYPE_MISMATCH: 주어진 key에 해당하는 자료구조가 Map이 아님.
- MEMCACHED_OVERFLOWED: Overflow 상태임. (overflowaction=error)

Map element를 upsert하는 예제는 아래와 같다.

``` c
void arcus_map_element_upsert(memcached_st *memc)
{
memcached_return_t rc;

uint32_t flags= 10;
uint32_t exptime= 600;
uint32_t maxcount= 1000;

memcached_coll_create_attrs_st attributes;
memcached_coll_create_attrs_init(&attributes, flags, exptime, maxcount);

// 1. CREATED_STORED
rc= memcached_mop_upsert(memc, "a_map", strlen("a_map"), "mkey", strlen("mkey"),
"value", strlen("value"), &attributes);
assert(MEMCACHED_SUCCESS == rc);
assert(MEMCACHED_CREATED_STORED == memcached_get_last_response_code(memc));

// 2. STORED
rc= memcached_mop_upsert(memc, "a_map", strlen("a_map"), "mkey1", strlen("mkey1"),
"value", strlen("value"), &attributes);
assert(MEMCACHED_SUCCESS == rc);
assert(MEMCACHED_STORED == memcached_get_last_response_code(memc));

// 3. REPLACED
rc= memcached_mop_upsert(memc, "a_map", strlen("a_map"), "mkey1", strlen("mkey1"),
"value", strlen("value"), &attributes);
assert(MEMCACHED_SUCCESS == rc);
assert(MEMCACHED_REPLACED == memcached_get_last_response_code(memc));
}
```

## Map Element 변경


Expand Down
27 changes: 23 additions & 4 deletions libmemcached/collection.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ typedef enum {
SOP_DELETE_OP,
SOP_EXIST_OP,
MOP_INSERT_OP,
MOP_UPSERT_OP,
MOP_UPDATE_OP,
MOP_GET_OP,
MOP_DELETE_OP,
Expand Down Expand Up @@ -65,6 +66,7 @@ static inline const char *coll_op_string(memcached_coll_action_t verb)
case SOP_DELETE_OP: return "sop delete ";
case SOP_EXIST_OP: return "sop exist ";
case MOP_INSERT_OP: return "mop insert ";
case MOP_UPSERT_OP: return "mop upsert ";
case MOP_UPDATE_OP: return "mop update ";
case MOP_GET_OP: return "mop get ";
case MOP_DELETE_OP: return "mop delete ";
Expand Down Expand Up @@ -105,6 +107,7 @@ static inline int coll_op_length(memcached_coll_action_t verb)
case SOP_DELETE_OP: return 11;
case SOP_EXIST_OP: return 10;
case MOP_INSERT_OP: return 11;
case MOP_UPSERT_OP: return 11;
case MOP_UPDATE_OP: return 11;
case MOP_GET_OP: return 8;
case MOP_DELETE_OP: return 11;
Expand Down Expand Up @@ -1291,7 +1294,7 @@ static memcached_return_t do_coll_insert(memcached_st *ptr,
{
/* no sub key */
}
else if (verb == MOP_INSERT_OP)
else if (verb == MOP_INSERT_OP || verb == MOP_UPSERT_OP)
{
size_t mkey_length = query->sub_key.mkey.length;
if (mkey_length > MEMCACHED_COLL_MAX_MOP_MKEY_LENG)
Expand Down Expand Up @@ -1334,7 +1337,8 @@ static memcached_return_t do_coll_insert(memcached_st *ptr,
if (attributes)
{
bool set_overflowaction= verb != SOP_INSERT_OP &&
verb != MOP_INSERT_OP
verb != MOP_INSERT_OP &&
verb != MOP_UPSERT_OP
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

아래 위치에 다음과 같은 코드가 있습니다.
MOP_UPSERT_OP인 경우도 동일하게 처리해야 하는 지를 검토 바랍니다.

      else if (rc == MEMCACHED_REPLACED && verb == BOP_UPSERT_OP)
      {
        /* bop upsert returns REPLACED if the same bkey element is replaced. */
        rc= MEMCACHED_SUCCESS;
      }

Copy link
Collaborator Author

@ing-eoking ing-eoking Jul 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOCS 확인 결과, 아래와 같습니다.

MEMCACHED_SUCCESS
  - MEMCACHED_STORED: 기존에 존재하던 B+tree에 element가 삽입됨.
  - MEMCACHED_CREATED_STORED: B+tree가 새롭게 생성되고 element가 삽입됨.
  - MEMCACHED_REPLACED: 기존에 존재하던 B+tree에서 동일한 bkey를 가진 element를 대체함.

MEMCACHED_REPLACED 또한 MEMCACHED_SUCCESS로 반환하게끔 되어있습니다.

mop upsert 또한 일치시키겠습니다.

&& attributes->overflowaction
&& attributes->overflowaction != OVERFLOWACTION_NONE;

Expand Down Expand Up @@ -1406,9 +1410,9 @@ static memcached_return_t do_coll_insert(memcached_st *ptr,
{
rc= MEMCACHED_SUCCESS;
}
else if (rc == MEMCACHED_REPLACED && verb == BOP_UPSERT_OP)
else if (rc == MEMCACHED_REPLACED && (verb == BOP_UPSERT_OP || verb == MOP_UPSERT_OP))
{
/* bop upsert returns REPLACED if the same bkey element is replaced. */
/* mop/bop upsert returns REPLACED if the same bkey element is replaced. */
rc= MEMCACHED_SUCCESS;
}
}
Expand Down Expand Up @@ -3793,6 +3797,21 @@ memcached_return_t memcached_mop_insert(memcached_st *ptr,
&query, NULL, attributes, MOP_INSERT_OP);
}

memcached_return_t memcached_mop_upsert(memcached_st *ptr,
const char *key, size_t key_length,
const char *mkey, size_t mkey_length,
const char *value, size_t value_length,
memcached_coll_create_attrs_st *attributes)
{
memcached_coll_query_st query;
memcached_mop_query_init(&query, mkey, mkey_length);
query.value = value;
query.value_length = value_length;

return do_coll_insert(ptr, key, key_length,
&query, NULL, attributes, MOP_UPSERT_OP);
}

memcached_return_t memcached_mop_update(memcached_st *ptr,
const char *key, size_t key_length,
const char *mkey, size_t mkey_length,
Expand Down
19 changes: 19 additions & 0 deletions libmemcached/collection.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -974,6 +974,25 @@ memcached_return_t memcached_mop_insert(memcached_st *ptr,
const char *value, size_t value_length,
memcached_coll_create_attrs_st *attributes);

/**
* Upsert (update the element if it exists, insert otherwise) an element into the map item.
* Optionally create the item if it does not exist.
* @param ptr memcached handle.
* @param key map item's key.
* @param key_length key length (number of bytes).
* @param mkey map element's key.
* @param mkey_length mkey length (number of bytes).
* @param value buffer holding the element value.
* @param value_length length of the element value (number of bytes).
* @param attrs if not NULL, create the item using the attributes if the item does not exist.
*/
LIBMEMCACHED_API
memcached_return_t memcached_mop_upsert(memcached_st *ptr,
const char *key, size_t key_length,
const char *mkey, size_t mkey_length,
const char *value, size_t value_length,
memcached_coll_create_attrs_st *attributes);

/**
* Update the existing element in the map item.
* @param ptr memcached handle.
Expand Down
69 changes: 69 additions & 0 deletions tests/mem_functions.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11402,6 +11402,74 @@ static test_return_t arcus_1_10_map_insert(memcached_st *memc)
return TEST_SUCCESS;
}

static test_return_t arcus_1_10_map_upsert(memcached_st *memc)
{
uint32_t flags= 10;
int32_t exptime= 600;
uint32_t maxcount= 1000;

memcached_coll_create_attrs_st attributes;
memcached_coll_create_attrs_init(&attributes, flags, exptime, maxcount);

// 1. CREATED_STORED
memcached_return_t rc= memcached_mop_upsert(memc, test_literal_param("map:a_map"), test_literal_param("mkey"), test_literal_param("value"), &attributes);
test_true_got(rc == MEMCACHED_SUCCESS || rc == MEMCACHED_BUFFERED, memcached_strerror(NULL, rc));
test_true_got(memcached_get_last_response_code(memc) == MEMCACHED_CREATED_STORED, memcached_strerror(NULL, memcached_get_last_response_code(memc)));

// 2. NOT_FOUND
rc= memcached_mop_upsert(memc, test_literal_param("map:no_map"), test_literal_param("mkey"), test_literal_param("value"), NULL);
test_true_got(rc == MEMCACHED_NOTFOUND, memcached_strerror(NULL, rc));

// 3. TYPE_MISMATCH
rc= memcached_set(memc, test_literal_param("kv:item"), test_literal_param("value"), 600, 0);
test_true_got(rc == MEMCACHED_SUCCESS || rc == MEMCACHED_BUFFERED, memcached_strerror(NULL, rc));
rc= memcached_mop_upsert(memc, test_literal_param("kv:item"), test_literal_param("mkey"), test_literal_param("value"), NULL);
test_true_got(rc == MEMCACHED_TYPE_MISMATCH, memcached_strerror(NULL, rc));

// 4. ELEMENT_REPLACED
rc= memcached_mop_upsert(memc, test_literal_param("map:a_map"), test_literal_param("mkey"), test_literal_param("new_value"), NULL);
test_true_got(rc == MEMCACHED_SUCCESS, memcached_strerror(NULL, rc));

memcached_coll_result_st result;
memcached_coll_result_create(memc, &result);
rc= memcached_mop_get(memc, test_literal_param("map:a_map"), test_literal_param("mkey"), false, false, &result);
assert(0 == strcmp("new_value", memcached_coll_result_get_value(&result, (size_t)0)));
memcached_coll_result_free(&result);

// 5. CLIENT_ERROR too large value
char too_large[MEMCACHED_COLL_MAX_ELEMENT_SIZE + 1];
rc= memcached_mop_upsert(memc, test_literal_param("map:a_map"), test_literal_param("mkey1"), too_large, MEMCACHED_COLL_MAX_ELEMENT_SIZE+1, &attributes);
test_true_got(rc == MEMCACHED_CLIENT_ERROR, memcached_strerror(NULL, rc));

sleep(MEMCACHED_SERVER_FAILURE_RETRY_TIMEOUT + 1);

// 6. OVERFLOWED
for (uint32_t i=1; i<maxcount; i++)
{
char buffer[15];
size_t buffer_len= snprintf(buffer, 15, "mkey%d", i);
rc= memcached_mop_upsert(memc, test_literal_param("map:a_map"), buffer, buffer_len, test_literal_param("value"), &attributes);
test_true_got(rc == MEMCACHED_SUCCESS || rc == MEMCACHED_BUFFERED, memcached_strerror(NULL, rc));
test_true_got(memcached_get_last_response_code(memc) == MEMCACHED_STORED, memcached_strerror(NULL, memcached_get_last_response_code(memc)));
}

memcached_coll_attrs_st attrs;
memcached_coll_attrs_init(&attrs);
memcached_coll_attrs_set_overflowaction(&attrs, OVERFLOWACTION_ERROR);
memcached_coll_attrs_set_expiretime(&attrs, 1000);

rc= memcached_set_attrs(memc, test_literal_param("map:a_map"), &attrs);
test_true_got(rc == MEMCACHED_SUCCESS, memcached_strerror(NULL, rc));

rc= memcached_mop_upsert(memc, test_literal_param("map:a_map"), test_literal_param("mkey1"), test_literal_param("value"), &attributes);
test_true_got(rc == MEMCACHED_SUCCESS, memcached_strerror(NULL, rc));

rc= memcached_mop_upsert(memc, test_literal_param("map:a_map"), test_literal_param("last_mkey"), test_literal_param("value"), &attributes);
test_true_got(rc == MEMCACHED_OVERFLOWED, memcached_strerror(NULL, rc));

return TEST_SUCCESS;
}

static test_return_t arcus_1_10_map_update(memcached_st *memc)
{
uint32_t flags= 10;
Expand Down Expand Up @@ -11910,6 +11978,7 @@ test_st arcus_1_9_tests[] ={
test_st arcus_1_10_tests[] ={
{"arcus_1_10_map_create", true, (test_callback_fn*)arcus_1_10_map_create},
{"arcus_1_10_map_insert", true, (test_callback_fn*)arcus_1_10_map_insert},
{"arcus_1_10_map_upsert", true, (test_callback_fn*)arcus_1_10_map_upsert},
{"arcus_1_10_map_update", true, (test_callback_fn*)arcus_1_10_map_update},
{"arcus_1_10_map_delete", true, (test_callback_fn*)arcus_1_10_map_delete},
{"arcus_1_10_map_delete_all", true, (test_callback_fn*)arcus_1_10_map_delete_all},
Expand Down
Loading