String and text fields coming back with over 100 invisible characters

[BennyHinrichs]

I cloned this repo yesterday https://github.com/sanity-io/sanity-template-remix-clean. Got some documents set up and querying in the client app. The response on the client for string and text fields includes an insane amount of non-visible characters like zero-width space and zero-width non-joiner. Copy the below example and paste it in your console to see.

"Always drink your ovaltine​​​​‌‍​‍​‍‌‍‌​‍‌‍‍‌‌‍‌‌‍‍‌‌‍‍​‍​‍​‍‍​‍​‍‌​‌‍​‌‌‍‍‌‍‍‌‌‌​‌‍‌​‍‍‌‍‍‌‌‍​‍​‍​‍​​‍​‍‌‍‍​‌​‍‌‍‌‌‌‍‌‍​‍​‍​‍‍​‍​‍‌‍‍​‌‌​‌‌​‌​​​‍‍​‍​‍‌‍​‌‍‌‍​‌‍​‌‌‍​‌‍‍​‌‍‌​‌‌​​‍‍​​​​​​​​​‍‌‍‍‌‌‍‍‌‌​‌‍‌‌‌‍‍‌‌​​‍‌‍‌‌‌‍‌​‌‍‍‌‌‌​​‍‌‍‌‌‍‌‍‌​‌‍‌‌​‌‌​​‌​‍‌‍‌‌‌​‌‍‌‌‌‍‍‌‌​‌‍​‌‌‌​‌‍‍‌‌‍‌‍‍​‍‌‍‍‌‌‍‌​​‌​​‌‍‌‌​‌​​‌‌‍‌​​​‍​‍​​‍‌​‍‌​‌‍​‌‍‌‍​​‌​​‍‌​‌​‌‍​‍​‌‌​‌‍​‍‌‌‍​‍​‌​‌‍​‍​​‌​‍‌‌‍​​‌​​‍‌​​​‌​‍‌‌‍​‌‌‍‌‍‌‍‌​‌‍​‌​‍‌​‌​‍‌‌​‌‍‌‌​​‌‍‌‌​‌‌‍‌‌​‍‌‍‌‌‌‌​​​‍‌​​‌‍​‌‌‌​‌‍‍​​‌‌‍‌‌‍‌‌‌​‌​‌‍​‌‌‍‌‌‍‌‌​‌‍​‍‌‍​‌‌​‌‍‌‌‌‌‌‌‌​‍‌‍​​‌‌‍‍​‌‌​‌‌​‌​​​‍‌‌​​‌​​‌​‍‌‌​​‍‌​‌‍​‍‌‌​​‍‌​‌‍‌‍​‌‍‌‍​‌‍​‌‌‍​‌‍‍​‌‍‌​‌‌​​‍‌‌​​‌​​‌​​​​​​​​​‍‌‍‌‍‍‌‌‍‌​​‌​​‌‍‌‌​‌​​‌‌‍‌​​​‍​‍​​‍‌​‍‌​‌‍​‌‍‌‍​​‌​​‍‌​‌​‌‍​‍​‌‌​‌‍​‍‌‌‍​‍​‌​‌‍​‍​​‌​‍‌‌‍​​‌​​‍‌​​​‌​‍‌‌‍​‌‌‍‌‍‌‍‌​‌‍​‌​‍‌​‌​‍‌‍‌‌​‌‍‌‌​​‌‍‌‌​‌‌‍‌‌​‍‌‍‌‌‌‌​​​‍‌‍‌​​‌‍​‌‌‌​‌‍‍​​‌‌‍‌‌‍‌‌‌​‌​‌‍​‌‌‍‌‌‍‌‌​‍​‍‌‌"

Other fields that are ostensibly strings but nominally have a different type (e.g. url) do not have the junk string appended to them.

So if I have a field like

defineField({
  name: 'path',
  title: 'Path to Resource',
  type: 'string',
  placeholder: 'user/profile',
})

It's more or less useless because when I consume it like

<a href={data.path}>{data.name}</a>

It just has a ton of garbage on the end. And sure, I can write a sanitizer function. But I feel like I shouldn't have to sanitize Sanity in a self-contained app. Anyone know why all the invisible characters are getting tagged on my strings?

[karltaylor]

@BennyHinrichs Unfortunately cannot help but just wanted to bump this to say we are also getting some invisible characters in our strings which is breaking out frontend 🤔.

Note: This only happens when we are on Presentation mode with Sanity Overlays

defineField({
        title: "Cover Media: Orientation",
        name: "mediaOrientation",
        type: "string",
        validation: (Rule) => Rule.required(),
        options: {
          list: [
            { title: "Landscape", value: "horizontal" },
            { title: "Portrait", value: "vertical" },
          ],
          layout: "dropdown",
        },
        initialValue: "horizontal",
      }),

When I get this value on the frontend, it looks harmless, but value === "horizontal" was returning false and driving us crazy, when we copy/pasted the string value we realised what was was happening.

[BennyHinrichs]

This is the sanitization function I ended up creating

export function sanitize(data: Record<string, any>) {
  Object.keys(data).forEach((key) => {
    if (typeof data[key] === 'string') {
      data[key] = data[key].replace(/(\u200b|\u200c|\u200d|\ufeff)/g, '');
    } else if (typeof data[key] === 'object') {
      sanitize(data[key]);
    }
  });
}

[paul-vd]

I just encountered the same issue for the following definition

    defineField({
      name: 'display',
      type: 'string',
      title: 'Display as',
      description: 'How should we display these images?',
      options: {
        list: [
          {title: 'Stacked on top of each other', value: 'stacked'},
          {title: 'In-line', value: 'inline'},
          {title: 'Carousel', value: 'carousel'},
        ],
        layout: 'radio',
      },
      initialValue: 'inline',
    }),
    ```

[paul-vd]

I believe this is stega encodings: https://www.sanity.io/docs/stega using stegaClean worked