Polish

- Finishing touches to CHANGELOG.md - Prettier formatting - Adding "Upgrade to v3" menu
tywalch · Oct 20, 2024 · 6b636ba · 6b636ba
1 parent 5f35339
commit 6b636ba
Show file tree

Hide file tree

Showing 8 changed files with 90 additions and 42 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -540,10 +540,10 @@ All notable changes to this project will be documented in this file. Breaking ch
 
 ## [3.0.0] 
 ### Changed
-- `{ compare?: 'keys' | 'attributes' }`
-- `validate` attribute callback now must return boolean. Return `true` for "valid" and `false` for "invalid"
-- `limit` now only applies a `Limit` parameter and does not play a role in pagination
+- ElectroDB is changing how it generates query parameters to give more control to users. Prior to `v3`, query operations that used the `gt`, `lte`, or `between` methods would incur additional post-processing, including additional filter expressions and some sort key hacks. The post-processing was an attempt to bridge an interface gap between attribute-level considerations and key-level considerations. Checkout the GitHub issue championed by @rcoundon and @PaulJNewell77 [here](https://github.com/tywalch/electrodb/issues/228) to learn more. With `v3`, ElectroDB will not apply post-processing to queries of any type and abstains from adding implicit/erroneous filter expressions to queries _by default_. This change should provide additional control to users to achieve more advanced queries, but also introduces some additional complexity. There are many factors related to sorting and using comparison queries that are not intuitive, and the simplest way to mitigate this is by using additional [filter expressions](https://electrodb.dev/en/queries/filters/) to ensure the items returned will match expectations. To ease migration and adoption, I have added a new execution option called `compare`; To recreate `v2` functionality without further changes, use the execution option `{ compare: "v2" }`. This value is marked as deprecated and will be removed at a later date, but should allow users to safely upgrade to `v3` and experiment with the impact of this change on their existing data. The new `compare` option has other values that will continue to see support, however; to learn more about this new option, checkout [Comparison Queries](https://electrodb.dev/en/queries/query#comparison-queries).
+- The `validate` callback on attributes now expects a strict return type of `boolean`. Additionally, the semantic meaning of a boolean response has _flipped_. The callback should return `true` for "valid" values and `false` for "invalid" values. If your validation function throws an error, ElectroDB will still behave as it previously did in `v2`, by catching and wrapping the error.
+- Providing the execution option `limit` on queries now _only_ applies a `Limit` parameter to its request to DynamoDB. Previously, the `limit` option would cause ElectroDB to effectively "seek" DynamoDB until the limit was _at least_ reached. The execution option `count` can be used in similar cases where `limit` was used, but performance may vary depending on your data and use case.
 ### Removed
-- `includeKeys` and `raw` execution options removed
+- The execution options `includeKeys` and `raw` were deprecated in version `2.0.0` and have now been removed in favor of the execution option `data`. To migrate from `v2`, use the options `{ data: "includeKeys" }` and `{ data: "raw" }` respectively.
 ### Fixed
 - Response typing and formatting logic for `delete` 
diff --git a/src/clauses.js b/src/clauses.js
@@ -122,10 +122,18 @@ let clauses = {
             state.setSK(composites);
             state.beforeBuildParams(({ options, state }) => {
               const accessPattern =
-                entity.model.translations.indexes.fromIndexToAccessPattern[state.query.index];
+                entity.model.translations.indexes.fromIndexToAccessPattern[
+                  state.query.index
+                ];
 
-              if (options.compare === ComparisonTypes.attributes || options.compare === ComparisonTypes.v2) {
-                if (!entity.model.indexes[accessPattern].sk.isFieldRef && sk.length > 1) {
+              if (
+                options.compare === ComparisonTypes.attributes ||
+                options.compare === ComparisonTypes.v2
+              ) {
+                if (
+                  !entity.model.indexes[accessPattern].sk.isFieldRef &&
+                  sk.length > 1
+                ) {
                   state.filterProperties(FilterOperationNames.eq, {
                     ...unused,
                     ...composites,
@@ -935,7 +943,10 @@ let clauses = {
             state.setSK(state.buildQueryComposites(facets, sk));
 
             state.whenOptions(({ options, state }) => {
-              if (options.compare === ComparisonTypes.attributes || options.compare === ComparisonTypes.v2) {
+              if (
+                options.compare === ComparisonTypes.attributes ||
+                options.compare === ComparisonTypes.v2
+              ) {
                 if (sk.length > 1) {
                   state.filterProperties(FilterOperationNames.eq, {
                     ...unused,
@@ -1003,7 +1014,10 @@ let clauses = {
           .setType(QueryTypes.between)
           .setSK(startingSk.composites)
           .beforeBuildParams(({ options, state }) => {
-            if (options.compare === ComparisonTypes.attributes || options.compare === ComparisonTypes.v2) {
+            if (
+              options.compare === ComparisonTypes.attributes ||
+              options.compare === ComparisonTypes.v2
+            ) {
               if (!entity.model.indexes[accessPattern].sk.isFieldRef) {
                 state.filterProperties(
                   FilterOperationNames.lte,
@@ -1063,7 +1077,10 @@ let clauses = {
           );
           state.setSK(composites);
           state.beforeBuildParams(({ options, state }) => {
-            if (options.compare === ComparisonTypes.attributes || options.compare === ComparisonTypes.v2) {
+            if (
+              options.compare === ComparisonTypes.attributes ||
+              options.compare === ComparisonTypes.v2
+            ) {
               const accessPattern =
                 entity.model.translations.indexes.fromIndexToAccessPattern[
                   state.query.index
@@ -1102,8 +1119,13 @@ let clauses = {
             );
             if (options.compare === ComparisonTypes.attributes) {
               const accessPattern =
-                entity.model.translations.indexes.fromIndexToAccessPattern[state.query.index];
-              if (!entity.model.indexes[accessPattern].sk.isFieldRef && attributes.sk.length > 1) {
+                entity.model.translations.indexes.fromIndexToAccessPattern[
+                  state.query.index
+                ];
+              if (
+                !entity.model.indexes[accessPattern].sk.isFieldRef &&
+                attributes.sk.length > 1
+              ) {
                 state.filterProperties(FilterOperationNames.gte, composites, {
                   asPrefix: true,
                 });
@@ -1136,7 +1158,9 @@ let clauses = {
           state.beforeBuildParams(({ options, state }) => {
             if (options.compare === ComparisonTypes.attributes) {
               const accessPattern =
-                entity.model.translations.indexes.fromIndexToAccessPattern[state.query.index];
+                entity.model.translations.indexes.fromIndexToAccessPattern[
+                  state.query.index
+                ];
               if (!entity.model.indexes[accessPattern].sk.isFieldRef) {
                 state.filterProperties(FilterOperationNames.lt, composites, {
                   asPrefix: true,
@@ -1169,7 +1193,10 @@ let clauses = {
           state.setSK(composites);
 
           state.beforeBuildParams(({ options, state }) => {
-            if (options.compare === ComparisonTypes.attributes || options.compare === ComparisonTypes.v2) {
+            if (
+              options.compare === ComparisonTypes.attributes ||
+              options.compare === ComparisonTypes.v2
+            ) {
               const accessPattern =
                 entity.model.translations.indexes.fromIndexToAccessPattern[
                   state.query.index
@@ -1502,7 +1529,8 @@ class ChainState {
 
   applyFilter(operation, name, values, filterOptions) {
     if (
-      (FilterOperationNames[operation] !== undefined) && (name !== undefined) &&
+      FilterOperationNames[operation] !== undefined &&
+      name !== undefined &&
       values !== undefined
     ) {
       const attribute = this.attributes[name];

diff --git a/src/client.js b/src/client.js
@@ -1,5 +1,5 @@
-const lib = require('@aws-sdk/lib-dynamodb')
-const util = require('@aws-sdk/util-dynamodb')
+const lib = require("@aws-sdk/lib-dynamodb");
+const util = require("@aws-sdk/util-dynamodb");
 const { isFunction } = require("./validations");
 const { ElectroError, ErrorCodes } = require("./errors");
 const DocumentClientVersions = {
@@ -11,12 +11,12 @@ const unmarshallItem = (value) => {
   const unmarshall = util.unmarshall || ((val) => val);
   try {
     value.Item = unmarshall(value.Item);
-  } catch(err) {
-    console.error('Internal Error: Failed to unmarshal input', err);
+  } catch (err) {
+    console.error("Internal Error: Failed to unmarshal input", err);
   }
 
   return value;
-}
+};
 
 const v3Methods = ["send"];
 const v2Methods = [

diff --git a/src/conversions.js b/src/conversions.js
@@ -3,23 +3,23 @@ function createConversions(entity) {
     fromComposite: {
       toKeys: (composite, options = {}) =>
         entity._fromCompositeToKeys({ provided: composite }, options),
-        toCursor: (composite) =>
+      toCursor: (composite) =>
         entity._fromCompositeToCursor(
           { provided: composite },
           { strict: "all" },
         ),
     },
     fromKeys: {
       toCursor: (keys) => entity._fromKeysToCursor({ provided: keys }, {}),
-        toComposite: (keys) => entity._fromKeysToComposite({ provided: keys }),
+      toComposite: (keys) => entity._fromKeysToComposite({ provided: keys }),
     },
     fromCursor: {
       toKeys: (cursor) => entity._fromCursorToKeys({ provided: cursor }),
-        toComposite: (cursor) =>
+      toComposite: (cursor) =>
         entity._fromCursorToComposite({ provided: cursor }),
     },
     byAccessPattern: {},
-  }
+  };
 
   for (let accessPattern in entity.model.indexes) {
     let index = entity.model.indexes[accessPattern].index;
@@ -65,4 +65,4 @@ function createConversions(entity) {
 
 module.exports = {
   createConversions,
-}
+};
diff --git a/src/entity.js b/src/entity.js
@@ -693,7 +693,11 @@ class Entity {
 
       response = this.formatResponse(response, parameters.IndexName, {
         ...config,
-        data: shouldHydrate && (!config.data || config.data === DataOptions.attributes) ? 'includeKeys' : config.data,
+        data:
+          shouldHydrate &&
+          (!config.data || config.data === DataOptions.attributes)
+            ? "includeKeys"
+            : config.data,
         ignoreOwnership: shouldHydrate || config.ignoreOwnership,
       });
       if (config.data === DataOptions.raw) {
@@ -1672,7 +1676,9 @@ class Entity {
         } else {
           throw new e.ElectroError(
             e.ErrorCodes.InvalidOptions,
-            `Invalid value for query option "compare" provided. Valid options include ${u.commaSeparatedString(Object.keys(ComparisonTypes))}, received: "${option.compare}"`,
+            `Invalid value for query option "compare" provided. Valid options include ${u.commaSeparatedString(
+              Object.keys(ComparisonTypes),
+            )}, received: "${option.compare}"`,
           );
         }
       }
@@ -1765,15 +1771,17 @@ class Entity {
         if (!DataOptions[option.data]) {
           throw new e.ElectroError(
             e.ErrorCodes.InvalidOptions,
-            `Query option 'data' must be one of ${u.commaSeparatedString(Object.keys(DataOptions))}.`,
+            `Query option 'data' must be one of ${u.commaSeparatedString(
+              Object.keys(DataOptions),
+            )}.`,
           );
         }
         config.data = option.data;
         switch (option.data) {
           case DataOptions.raw:
             config.raw = true;
             break;
-          case  DataOptions.includeKeys:
+          case DataOptions.includeKeys:
             config.includeKeys = true;
             break;
         }
@@ -2737,8 +2745,10 @@ class Entity {
 
     const customExpressions = {
       names: (queryOptions.expressions && queryOptions.expressions.names) || {},
-      values: (queryOptions.expressions && queryOptions.expressions.values) || {},
-      expression: (queryOptions.expressions && queryOptions.expressions.expression) || "",
+      values:
+        (queryOptions.expressions && queryOptions.expressions.values) || {},
+      expression:
+        (queryOptions.expressions && queryOptions.expressions.expression) || "",
     };
 
     let params = {
@@ -2888,7 +2898,10 @@ class Entity {
   _getComparisonOperator(comparison, skType, comparisonType) {
     if (skType === "number") {
       return Comparisons[comparison];
-    } else if (comparisonType === ComparisonTypes.attributes || comparisonType === ComparisonTypes.v2) {
+    } else if (
+      comparisonType === ComparisonTypes.attributes ||
+      comparisonType === ComparisonTypes.v2
+    ) {
       return KeyAttributesComparisons[comparison];
     } else {
       return Comparisons[comparison];
@@ -2922,8 +2935,10 @@ class Entity {
 
     let customExpressions = {
       names: (queryOptions.expressions && queryOptions.expressions.names) || {},
-      values: (queryOptions.expressions && queryOptions.expressions.values) || {},
-      expression: (queryOptions.expressions && queryOptions.expressions.expression) || "",
+      values:
+        (queryOptions.expressions && queryOptions.expressions.values) || {},
+      expression:
+        (queryOptions.expressions && queryOptions.expressions.expression) || "",
     };
 
     let keyExpressions = this._queryKeyExpressionAttributeBuilder(
@@ -3389,7 +3404,9 @@ class Entity {
         if (missingAttributes.length) {
           throw new e.ElectroError(
             e.ErrorCodes.IncompleteIndexCompositesAttributesProvided,
-            `Incomplete composite attributes provided for index ${index}. Write operations that include composite attributes, for indexes with a condition callback defined, must always provide values for every index composite. This is to ensure consistency between index values and attribute values. Missing composite attributes identified: ${u.commaSeparatedString(missingAttributes)}`,
+            `Incomplete composite attributes provided for index ${index}. Write operations that include composite attributes, for indexes with a condition callback defined, must always provide values for every index composite. This is to ensure consistency between index values and attribute values. Missing composite attributes identified: ${u.commaSeparatedString(
+              missingAttributes,
+            )}`,
           );
         }
 

diff --git a/src/schema.js b/src/schema.js
@@ -470,12 +470,14 @@ class Attribute {
           let isValid = !!definition(val);
           return [
             isValid,
-            isValid ? [] : [
-              new e.ElectroUserValidationError(
-                this.path,
-                "Invalid value provided",
-              ),
-            ]
+            isValid
+              ? []
+              : [
+                  new e.ElectroUserValidationError(
+                    this.path,
+                    "Invalid value provided",
+                  ),
+                ],
           ];
         } catch (err) {
           return [false, [new e.ElectroUserValidationError(this.path, err)]];

diff --git a/src/types.js b/src/types.js
@@ -7,7 +7,7 @@ const DataOptions = {
   raw: "raw",
   includeKeys: "includeKeys",
   attributes: "attributes",
-}
+};
 
 const BatchWriteTypes = {
   batch: "batch",

diff --git a/www/src/config.ts b/www/src/config.ts
@@ -57,6 +57,7 @@ export const SIDEBAR: Sidebar = {
         text: "Single-Table Relationships",
         link: "en/core-concepts/single-table-relationships",
       },
+      { text: "Upgrade to v3", link: "en/core-concepts/v3-migration" }
     ],
     "Data Modeling": [
       { text: "Entities", link: "en/modeling/entities" },