Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[templates] Ensure that template instantiation actually improves the platform #704

Open
smaug---- opened this issue Nov 10, 2017 · 39 comments

Comments

@smaug----
Copy link

Before adding anything as complicated as https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md it is better to ensure it improves the platform.
I'm especially interested in the performance aspect.

If a polyfill could implement the proposal without being significantly slower, would there be need to add the API natively?

@jakearchibald
Copy link

Aside from the performance aspect, @surma is interesting it writing developer-facing documentation for this proposal, to lower the bar for feedback.

@JanMiksovsky
Copy link

I spent last week implementing a polyfill for a syntax variation of the template instantiation proposal, and also ported a number of web components from the Elix library over to that polyfill to see whether template instantiation would make creating such components easier.

Upon reflection, it does not appear that template instantiation really helps us.

The web components we create are typically more complex than can be addressed with mustache syntax. We use a mixin architecture to create our components; e.g., a complex carousel component of ours is currently built from ~20 different mixins, each contributing some aspect of programmatic or interaction behavior to the component class.

Significantly, in this mixin architecture, the component class that defines the template doesn't know what properties of sub-elements within that template will need to be modified by the mixins. Instead, a component asks itself (and therefore the mixins along its prototype chain) to build up a dictionary of all the updates that should be applied to the elements in its shadow tree. Then the component applies those updates.

Such separation of concerns means we cannot use the mustache syntax proposal, because that requires the complete set of attributes that will be modified to be expressed directly in the template.

One thing that might help would be a way for a component class to indicate that a collection of property updates be applied to a given element. Perhaps this would be something along the lines of React's spread syntax:

<div {...props}/>

We'd want to avoid introducing new syntax, so maybe we'd have a special property?

<div properties="{{props}}"></div>

I had a chance to talk about these experiments with @rniwa at Apple last week. During our conversation, I proposed adding something like the properties= idea above to Ryosuke. He suggested that, given our existing mixin architecture, what we'd really benefit from is something that lets us apply a collection of property updates to a tree of elements.

I've gone ahead and written up that idea as a proposal for bulk property updates. (Filed as a separate issue.) That includes a hypothetical applyPropertiesById method that would achieve what Ryosuke suggested. With something like that, we wouldn't need the template instantiation proposal.

In any event, for the time being, we don't see template instantiation helping our web components library.

@rniwa
Copy link
Collaborator

rniwa commented Dec 19, 2017

It's true that much (if not all) of what this API provides can be implemented in JS. For us, the main benefit of this proposal is that it paves a way to come up with a declarative syntax for custom elements. It also provides a useful mechanism for template libraries to update different parts of DOM easily. Finally, the default template processor provides a mechanism to easily create a template instance without having to import a third party library.

@mildred
Copy link

mildred commented Jan 31, 2019

Hello, I'm new to this discussion, but I'd like to point out that there is no need for special syntax with curly braces to get useful things from templates. I'd like to point out two different template engines based on the same principles each that don't require any template syntax at all:

They both work on the same principles: You define your markup with absolutely no special syntax, custom attributes or anything like that. Then, you define a JSON object on the side that goes with your markup that is telling how to instantiate the markup for a given input data structure.

What I really like from this approach is that you get a clear separation between the markup and the data structure you want to template. From the beginning, web standards is all about separation of concerns, and I believe this approach fits very nicely with that.

@dmitriid
Copy link

@rniwa

The main problem with template instantiation proposal is that it attempts to solve a symptom, not provide the cure for the cause.

And there are two causes:

  • no declarative API for DOM APIs
  • no data binding mechanism

As all compromises, template instantiation proposal solves both poorly.

JS. For us, the main benefit of this proposal is that it paves a way to come up with a declarative syntax for custom elements.
It also provides a useful mechanism for template libraries to update different parts of DOM easily.

This "syntax" already exists, and has been battle proven over several years on thousands of web sites. As is the ability to efficiently and easily update different parts of the DOM.

And it looks more or less like this:

h('div', 
   { 
        attrs: { className: "x" }, 
        props: { duration: y }, 
        on: { click: () => z()  }
   }, 
   [ h(...), h(...) ]
);

Yes. It's virtual DOM, popularised by React, and which exists as virtual-dom, hyperscript etc. There are differences in the APIs between these libraries, but they are quite superficial, and they more or less converged on the same set of principles:

  • declarative API that is pure JS, no external dependencies, no templates, just functions
  • the API is essentially f( tag, { props, attrs, events, lifecycle }, [children]) which is easy to reason about, easy to analyse, debug, optimise
  • it solves about 100% of Web Component's data binding woes
  • it solves the whole "we don't know how to provide a declarative API for WebComponents" because it also solves "we will provide a unified consistent API for all of HTML/DOM, not just WebComponents"
  • it solves server-side rendering

Meanwhile templates... I don't know what actual issues do they solve? They introduce way more problems than it's worth:

  • somehow only limited to template instantiation inside custom elements. Why not use them for the rest of the DOM?

  • is a new templating language:

    • has no relation to the rest of the browser
    • needs a separate parser and execution model (not a small chunk of work in itself + an increased attack surface)
    • as all templating languages will be limited (so, limited control structures, limitations on what you can call from the host environment, poor and limited available standard library, if any)
    • only provides an imperative API to work with from JS that's no better and no easier than existing DOM APIs:
    // before
    var X = template.getElementById("#part");
    X.appendChild(document.createElement('span'));
    X.appendChild(document.createTextNode('hello'));
    
    // after
    var X = template.getRequiredTemplatePartSomehow();
    X.replace([document.createElement('span'), 'hello']);

    The vast majority of the web isn't even using DOM APIs with Web Components, and goes straight for .innerHtml (even lit-html does that with reckless abandon). Given that the template proposal doesn't really add any new easy-to-use APIs, it will still be easier to drop to .innerHTML or reach for any of the virtual dom libraries out there than use the templates.

In my opinion, the templating proposal has its value to show some the problems that people experience with Web Components, but can never be a solution. The solution lies in better, easier-to-use declarative DOM APIs.

@bahrus
Copy link

bahrus commented Mar 21, 2019

Hi @dmitriid

I don't follow why template instantiation could only be used inside a web component. It would be useful anytime there's repetitive markup that needs customizing in each instance. I encounter that scenario quite a bit, before and after web components.

The fundamental issue templates are trying to solve, as I understand it, is that repeatedly cloning a template for large chunks of easy to parse html is faster than making lots of appendChild or innerHTML calls, which can only be done after the expensive job of parsing the JavaScript. This is an empirical claim. Are you claiming otherwise? It would be great to show your counter-factual results.

HTML templates also feel much more "declarative" to me. Templates are inert, containing a data format (xml-ish) so they can be loaded quickly with no side effects. hyperscript may also have no side effects for simple examples (other than rendering the html of course), but you can evaluate any function you want during the processing, which could have unexpected side effects. To be fair, some aspects of the template instantiation proposal (which looks quite different from how you are describing it, with your before and after -- are we looking at the same proposal?) may also allow for functions with side effects, but I could be wrong.

For the record, I'm not opposed to introducing an h function into the api, if it fulfills some useful purpose, but I fail to see how it is easier to use than tagged template literals. Could you elaborate? The performance numbers I've seen comparing lit-html and hyperHTML, compared to react (and even preact) make me wonder what your objections are?

@dmitriid
Copy link

@bahrus

I do apologise if I sound terse or rude in the text below, as I'm writing this rather quickly in a spare moment (I wanted to acknowledge your response quickly, and not have you wait for a day or two).

I don't follow why template instantiation could only be used inside a web component. It would be useful anytime there's repetitive markup that needs customizing in each instance.

The whole discussion is mostly in the context of Custom Elements and Shadow DOM (see 2. Use Cases). The proposal itself is only limited to <template> elements.

That's why in my mind it was only limited to custom elements.

However, true, you can use them elsewhere:

rniwa = {name: "R. Niwa", email: "[email protected]"};
document.body.appendChild(contactTemplate.createInstance(rniwa));

This does still leave the question of how to more complex/nested templates where parts of a template are defined in other templates etc.

but you can evaluate any function you want during the processing, which could have unexpected side effects.

That is, side effects that are desired by the developer using it ;). There's no way to use a template that has this:

{{foreach items}}
   <li class={{class}} data-value={{value}}>{{label}}</li>
{{/foreach}}

without first calling arbitrary functions to create these items and binding them to the template. And since the proposed way of creating such things is the same old DOM API with hardly any improvements, I fail to see the improvement.

And as you correctly mentioned, templates will need to be able to call arbitrary functions.

I fail to see how it is easier to use than tagged template literals. Could you elaborate? The performance numbers I've seen comparing lit-html and hyperHTML, compared to react (and even preact) make me wonder what your objections are?

The only reason lit-html works as it does is that dozens (hundreds?) of engineers spent hundreds of hours optimising to things that are frequently used and abused in JS:

  • string processing and RegExps
  • .innerHtml

More or less the only thing that lit-html does is parse a string at runtime, with regexps, concatenate it into an opaque string blob, and dump it into browser via .innerHtml, and let browser deal with it. (I have a separate rant on tagged template literals elsewhere).

It's not a good thing. It's a very bad thing, it only happens to work fast enough because browsers have had decades to optimise this (and ten years ago using .innerHtml was considered extremely bad practice see e.g. this StackOverflow comment, and even now .innerHtml is not optimised enough for certain cases, and there are security considerations).

Meanwhile declarative DOM/virtual DOM libraries have to recreate the entire DOM model in memory, and manually diff it against the browser DOM.

The solution to all that (and to template instantiation) is definitely not, in my opinion:

  • a limited templating language that can only be used in <template> elements, and whose actual API is only marginally better than the rest of DOM APIs (and introduces a host of other problems, some of which
  • concatenating strings at runtime and dumping them into the DOM via .innerHTML

A declarative API (with, hopefully, browser-native DOM-diffing) solves a lot of the problems:

  • data binding (it just becomes function calls with your data/parameters)
  • efficient DOM updates (you only regenerate parts of the tree that change)
  • instantiation (it's just a function call, and many lifecycle methods to trigger it that are already there)

It may/will still be awkward to use, obviously. The next best thing, IMO, would be a standard/declarative way to create a DOM AST that you can pass to the browser. Virtual DOM libs, in essence, do that already. To quote Dan Abramov:

// JSX is a syntax sugar for these objects.
// <dialog>
//   <button className="blue" />
//   <button className="red" />
// </dialog>
{
  type: 'dialog',
  props: {
    children: [{
      type: 'button',
      props: { className: 'blue' }
    }, {
      type: 'button',
      props: { className: 'red' }
    }]
  }
}

Unfortunately, we cannot generate this AST for the browser and let it deal with it efficiently. We have to fall back to .innerHtml or hundreds of lines of tedious imperative code (or use libs/frameworks/wrappers).

With a natively supported declarative description of the DOM/AST you still have a low-level primitive that libs/frameworks can make even easier to use, but you can also trivially use it in vanilla JS code.

I do hope I made sense in the ramblings above :)

@bahrus
Copy link

bahrus commented Mar 21, 2019

Thanks, @dmitriid, for your civil and non-rambling response :-)

I agree templates are a key part of the web component stack, and in retrospect you were replying to a message arguing why an (apparently) small library like what template instantiation would (apparently) entail should be built in to the platform, and the argument was made that it would benefit declarative custom elements.

But more generally, I for one have taken the liberty of using templates without web components on a number of occasions, and I don't think I'm alone.

I actually agree with you that the platform would benefit from some helper functions for those scenarios where a programmatic api is needed -- one that improves upon Object.assign, one that is specifically tailored for setting properties / attributes / events on a DOM element, i.e. a similar purpose to what you are laying out.

For example, I've been toying with a function I call "decorate", which I modeled after Vue/Polymer 1, but now realize, thanks to your bringing it to my attention, is quite similar to h.

The difference is that this decorate function is applied to an existing element, coming from a template or existing DOM tree, rather than being only useful for generating the HTML data structure itself.

I do think, given the wide range of libraries doing something similar, having something built in to the platform would be useful. Just my two cents. But so would template instantiation. I don't see why one precludes the other.

If a template api/syntax has built in functions to do certain things (like for-each), i.e. officially sanctioned functions, that is quite different from saying you can use any user-defined function you want. For a concrete example of what I'm getting at, I need to change the subject slightly: Take the github developers who are managing the github web site-. They are defining web components, which can enhance the markdown vocabulary. I recently realized I can use their custom elements in my markdown! Github only trusts that because they trust their own web components. They won't allow us to use arbitrary web components, for fairly obvious reasons.

I'm not saying user-defined functions must be forbidden in template languages, only that it is a significant line one is crossing, one which could be use to separate "non-declarative" vs "declarative" in a coherent (I think) way.

I suspect the author of hyperHTML would take issue with the statement that it required dozens or hundreds of engineers to match the performance of h based libraries :-).

Thanks for the link to your critique of tagged template literals. I'll take a look.

@WebReflection
Copy link

I suspect the author of hyperHTML would take issue with the statement that it required dozens or hundreds of engineers to match the performance of h based libraries

I think @dmitriid was referring to the fact these libraries (lit, lighter, or hyperHTML) are fast only because the primitives used have been made fast by browsers engineers.

I also think there's no shame in knowing, and using, fast primitives to deliver better UX, and that's naturally the goal of any performance oriented abstraction anyway 👋

@bahrus
Copy link

bahrus commented Mar 24, 2019

@dmitriid, is that what you meant? If so, apologies for misinterpreting.

I guess I was thrown by the use of the word "to" in:

The only reason lit-html works as it does is that dozens (hundreds?) of engineers spent hundreds of hours optimising to things that are frequently used and abused in JS:

followed by his comment on what the browser engineers have done:

it only happens to work fast enough because browsers have had decades to optimise this.

I.e hundreds of lit-html engineers (I can't seem to locate his reference to hyperHTML, which line is that?) used bad practices, which happened to not matter because of the decades spent by browser engineers optimizing on those bad practices.

Apologies for my lack of reading comprehension.

I agree with you -- lit-html, lit, hyperHTML are fast and easy to use, great libraries, because they were built and optimized by a handful of dedicated and smart engineers, built on fast primitives built by great browser engineering teams. Somehow I didn't quite find that sentiment shared by @dmitriid, but what do I know? @dmitriid, we're all in agreement?

@Jamesernator
Copy link

@dmitriid Neither template instantiation or lit-html use .innerHTML to update the DOM. This seems to be some FUD around how template instantiation works but template instantiation and lit-html actually work more efficiently than a virtual DOM by holding references to part of the DOM that have template parts. In lit-html's case, it only creates the initial template using .innerHTML, to actually update the DOM it inserts nodes directly into the correct location (and similar for attributes).

For example consider this template:

<template id="exampleTemplate">
  <span title="{{foo}}">This is a sample {{bar}} with instantiation</span>
  <span>Also {{foo}}</span>
</template>

when we create an instance of the template a set of references are created to the elements that have that name:

{
  foo: [
    AttributeTemplatePart {
      el: /* ref to <span title="{{foo}}"> element */
      attrName: 'title',
    },
    NodeTemplatePart {
      previousNode: /* The previous text node before {{foo}}: "Also" */
      nextNode: /* The text node after {{foo}} */
    },
  ],
  bar: [
    NodeTemplatePart {
      previousNode: /* The previous text node before {{bar}}: "This is a sample " */
      nextNode: /* The next text node after {{bar}}: " with instantiation"
    }
  ],
}

now when we call something like instance.update({ foo: 'banana', bar: 'cabbage' }) all .update needs to do is something like this:

function update(data) {
  for (const [key, value] of Object.entries(data)) {
    for (const templatePart of this._templateParts[key]) {
      templatePart.update(value)
    }
    this._templateParts[key].update(value)
  }
}

this is way more efficient than maintaining a virtual copy of the DOM as we simply implement AttributeTemplatePart and NodeTemplatePart so that they go directly to their location in the DOM and update the value.

e.g. NodeTemplatePart could be implemented like this:

class NodeTemplatePart {
  constructor(previousNode, nextNode) {
    this._previousNode = previousNode
    this._nextNode = nextNode
  }

  // This is overly simplified and assumes there's a node both before and after
  // the {{curlies}}, a real implementation would hold a reference to the parent element
  // as well and if there's no previousNode/nextNode it'd just replace the whole contents
  update(values) {
    if (typeof values === 'string') {
      values = [new Text(values)]
    }
    while (this._previousNode.nextNode !== this._nextNode) {
      this._previousNode.nextNode.remove()
    }

    for (const value of values) {
      this._nextNode.parentNode.insertBefore(value, this._nextNode)
    }
  }
}

@WebReflection
Copy link

WebReflection commented Mar 25, 2019

Just for documentation and clarification sake, everything @Jamesernator said is the exact same for both lighterhtml and hyperHTML, based indeed on the same domdiff library, which uses innerHTML once per unique template tag, on an offline template element, and update the related DOM when needed, it never uses innerHTML again (unless explicitly required by the user, but that's another story)

@dmitriid
Copy link

dmitriid commented Apr 1, 2019

I think we've steered slightly off-track, so I'll try (hopefully :) ) to get back to what I was intended to say (once again, sorry if I get sidetracked again).

There are two parts to this long-winded comment:

  1. What I think (and what we should still strive for)
  2. What may actually need

What I think (and what we should still strive for)

In a tl;dr kind of way my thinking comes to this:

lighterhtml and hyperHTML, based indeed on the same domdiff library, which uses innerHTML once per unique template tag

There are two question arising:

  1. In 2019 why is there no better API than dumping blobs of strings into DOM via .innerHtml?
  2. Why are facilities like domdiff not provided natively by the platform?

In my opinion, if we answer and find solutions to these two questions, the entire template instantiation proposal will be rendered moot. To slightly re-word the reasoning behind the proposal:

The HTML5 specification ... doesn't provide a native mechanism to instantiate with some parts of it substituted, conditionally included, or repeated based on JavaScript values

And then it does correctly say:

making it hard for web developers to combine otherwise reusable components when they use different templating libraries.

The thing is though: is the answer to that a yet another incompatible templating library and syntax? Developers will continue using their own incompatible templating libraries regardless. The reason is simple: any and all templates are limited in functionality and scope. Incompatible templating systems arise because people find that some templating system X is lacking some crucial functionality.

You can see it with Web Components themselves. When they were finalised and started shipping in browsers there was abundant joy. However, just two short years later, the mood has shifted to "web components APIs are a barebones set of low-level primitives aimed at library writers" and people go out of their way to create better more useful abstractions on top. Including pushing everything into strings.

But this was a sidetrack. Let's get back to template instantiation. Among other things mentioned (or shown as use cases):

  • it's hard to create DOM via DOM APIs
  • it's hard to find and update required parts of the DOM
  • it's hard to do multiple updates efficiently
  • it's hard to compose actual components (where a component maybe a deeply nested, updatable DOM structure)

The reason is obvious: these things are hard to do because DOM APIs are low level, imperative and verbose. However, it has been proven multiple times that these APIs can be made sufficiently high-level, declarative and succinct even in userland. And that they literally solve all of the problems above:

  • create DOM? A breeze with declarative APIs
  • find an update required parts? Same
  • multiple updates? Ditto
  • compose? These are just nested function calls/ASTs, we compose things all the time

Instead of providing these facilities, the proposal gives us:

  • a new templating language (incompatible with everything else)
  • that is both very limited in functionality (poorly specified/underspecified if and foreach as the only control structures) and unlimited in functionality (we can call functions in the host language)
  • that is alien to the system (literally nothing in the platform looks or behaves like this)
  • that requires a different parser, a different execution model, hooks into the rest of the platform to make it workable
  • solves the main use case of composing components poorly, or not at all (you need to nest template elements within template elements within template elements...)
  • doesn't specify how you would instantiate elements not from static sources, but from dynamic data (I'm guessing .innerHtml again?)

Even though all these problems have been solved dozens of times over by simply providing a better API.

It is my continuing belief that the only right way forward is not to make the platform increasingly weird and complex, but to improve the APIs available to developers and let them figure out what to do with them.

See part 2 on what developers already do with existing APIs and how we can help them by being a better platform.

Declarative DOM tree

As I already mentioned a few thousand times :) developers already create DOM trees, diff them, and apply only changes. In userland. Why can't platform itself provide similar APIs and capabilities is beyond my understanding at this point.

However, there are now emerging technologies that may require not just native browser APIs but also a way to provide the browser with a DOM Tree and let the browser figure out what to do. I'm talking about Phoenix LiveView. The idea is as follows:

  • on the server figure out the actual data that has changed
  • send it to the browser
  • use morphdom to figure out minimal updates to the DOM
  • update DOM

In the end it does this: ezgif-3-7e1ae7100bef

(For crazy versions, see server-rendered FlappyBird or search Twitter for LiveView).

Template instantiation and updates would have hard time offering anything of value to an approach like this. A friendly native API to tell the browser exactly what you need or a way to declaratively define and provide DOM trees though?

  • They would basically kill the need for all the VDOM implementations (except tiny wrappers for people who don't like the original API)
  • They would obviate the need for a complex solution like template instantiation
  • They would actually provide better interoperability because whatever templating system, or a DOM library you have, if in the end of the day it outputs the same DOM structures (or uses the same APIs), it doesn't matter what it looks like. For example, Vue's templates compile to code that is for all intents and purposes indistinguishable from what React's JSX compiles to. And just because these are two different userland implementations they are (somewhat) incompatible. Why not provide a common API in the platform?
  • They would let people experiment even more freely with more crazy ideas (60 fps animation entirely provided by the server is already possible, see LiveView links above) because they wouldn't be fighting the limitations of the platform, but be empowered by it.

In my opinion template instantiation and the current (and the only, cca 90s) generation of DOM APIs provide none of that and are very reluctant to move forward.

@bahrus
Copy link

bahrus commented Apr 1, 2019

Hi @dmitriid,

Thanks for helping get the conversation back on track (though I did think the previous two comments were quite on-track and informative as well).

In 2019 why is there no better API than dumping blobs of strings into DOM via .innerHtml?

Excellent question! I wish with all my heart that HTML Modules / Imports would have received, in conjunction / parallel with ES Module imports, the same degree of attention the past few years that ES Modules did. But I guess, bowing to the fact that JS imports was in higher demand (an example, in my mind, of questionable coding practices begetting questionable priorities from the standards committees), HTML Modules (and people like me who prefer to send data in their native data format) took a back seat.

But I have good news for you -- with the HTML Modules proposal which is hopefully going to ship soon, it won't be necessary to use .innerHTML to create a template object ready for fast cloning. The server can send a tag with the HTML that needs rapid cloning directly in the document! No API needed at all! But I disagree with you that that answer makes template instantiation less useful -- quite the opposite.

What I'm not yet picking up from you is a recognition of what templates bring to the table -- the ability to quickly clone a chunk of HTML, and add it to the tree using appendChild. I.e the ability to work with repeating data, like grids or virtual lists (or multiple instances of the same web components) efficiently.

I do share the general hope with you that, should template instantiation / and rapid updates of the cloned DOM tree ship in the browser, that whatever low-level functions are needed to do that in as fast a manner as possible -- a DOM Diff engine of sorts, in other words -- that where possible, (some of) those low-level functions might also be added to the platform, which could benefit other scenarios, such as scenarios where the original HTML didn't come from a template, but may have instead come from server-side rendered HTML. I.e., tell the engine via an api where to locate the "parts" for rapid updates. Scenarios where the HTML isn't very repetitive, which doesn't exhibit the kind of repetitive structures templates are designed to optimize. Or scenarios where a client-side API is used to generate the entire HTML structure without the help of templates (hard as it is for me to imagine why that would be the right approach).

@rniwa
Copy link
Collaborator

rniwa commented Apr 2, 2019

@dmitriid: Thanks for a very insightful feedback. I love when people give very constructive & detailed feedback based on use cases & case studies like that.

As I stated somewhere before, our approach to template instantiation broadly has three goals:

  1. Provide a high performance primitive for template engines & frameworks to build on top of.
  2. Provide a minimal high-level template support so that authors don't necessarily rely on a library or a framework to make a good use of templates
  3. Provide a common framework by which different template engines & frameworks can work together

I understand that the need for (1) is quite high, and your criticism of the original proposal is understandable because it had many flaws. However, that is the point of making a proposal, and we're happy to work with you and the rest of web developer community to figure out the right set of primitives and API surface to add to the platform.

Template instantiation and updates would have hard time offering anything of value to an approach like this. A friendly native API to tell the browser exactly what you need or a way to declaratively define and provide DOM trees though?

That's more or less NodeTemplatePart and AttributeTemplatePart aim to provide. When you set a new value to a template part, the browser can figure out whether the change actually needs to propagate to the node or not by figuring out the value change. We didn't include any node diff'ing algorithm into the proposal but it could be done.

We've been working with @justinfagnani from Google to improve our proposal ever since we made the initial proposal, but it would be really helpful to document what you consider to be the ideal API. For example, @WebReflection's proposal about DocumentFragment in whatwg/dom#736 has been very useful for us because that's more or less where I started before I eventually warmed up to the idea of NodeTemplatePart.

@LucaColonnello
Copy link

I read all of this, and although I'm very excited about Template Instantiation, I agree with @dmitriid that it's quite high level, bring mustache syntax, often use as a top abstraction template engine, as the only solution for truly dynamic HTML.

If there was something like a native patch api, this wouldn't have been required, as the template could have been generated by any abstraction.

I'd like to emphasise the need for this sort of API cause since 10 years, developers have come up with different solutions and abstraction for this very problem...

DOMChangeList API seems to offer a similar approach to what Template Instance does, in the sense that once you figure out changes, you can delegate that to a low-level DOM API that already applies them for you.

My personal experience working with JSX at least, us that it's very nice to be able to use JS in a declarative way to build your template. You don't need to learn any new DL and it just works, letting you write template logic as you please.

The problem I see with a DOM diffing low-level API is only one.
As mentioned above, you don't always need DOM diffing. You could create a DOMChangeList based on the amount of changes coming from state changes. For specific cases, that would be more efficient.

So, I do agree that we require a way for the underlying API to know where changes will happen in the document. In that way, the low-level API can choose which technique is best to apply, based on his knowledge of the template and where changes are.

Ultimately, I disagree in general with statements like 'Template Instantiation does not bring anything to the table, it does add much', just because of what I wrote above...

But I don't like for the native standard to choose the template syntax you should use.
If there was a way to change the proposal to support JS instead of mustache, I don't know maybe a template strings like solution...

@dmitriid
Copy link

dmitriid commented May 5, 2019

I realised I've been absent from this discussion for quite some time. But, you know, there's always someone wrong on the internet, so my attention got diverted :)

I realised the main reason we don't agree on what template instantiation. Template instantiation presumes that the template already exists. That is, it's a static chunk of HTML (as a string?) with some dynamic parts described using the moustache syntax. And then that static chunk gets applied to the document.

See for yourself (I'll use the quote format to present all my arguments as a single chunk):

with the HTML Modules proposal which is hopefully going to ship soon, it won't be necessary to use .innerHTML to create a template object ready for fast cloning.

HTML Modules assumes there's already some static HTML defined and ready for import

what templates bring to the table -- the ability to quickly clone a chunk of HTML, and add it to the tree using appendChild.

This assumes that the HTML chunk already exists

That's more or less NodeTemplatePart and AttributeTemplatePart aim to provide. When you set a new value to a template part, the browser can figure out whether the change actually needs to propagate to the node

This also assumes that those parts are already there, and inserted into the document

However, the only reasonable API for creating those templates? Probably .innerHTML :)

So, in my mind, the template instantiation proposal for some reason separates creating and updating dynamic HTML into several stages:

  • somehow create the template (how?)
  • somehow insert it into the document or into another template (how?)
  • when values change, call .update, and the values will be updated (the proposal)

However, from the point of view of most developers (yup, speaking for everyone here :) ) there's only one stage: given a certain state, update the DOM.

A common example: Load a list of items. While they are being loaded, display "Loading". Once they are loaded, display them.

In most templating systems, or in any JS-only approach like React's JSX, it's the same thing:

// using JSX
return <div>
   {{ loading ? 'Loading...' : items.map(i => `<div>${i.name}</div>`}) }}
</div>

Template Instantiation as it currently stands has no support for this use case:

  • conditionals "can be implemented by libraries and frameworks" (so, not a part of the proposal)
  • loops have at least three different solutions, all with their drawbacks
  • in all cases the developer would have to create a separate template just for the list of items to be able to display them instead of "Loading...". Which comes back to "how?" :) And how will the get into the original template in the first place (.update or .appendChild?)

And that's just for the most simple use case.


I also doubt these new APIs will gain any significant adoption as "a common set of primitives for various libs and frameworks to work together". To begin with, there already is a common set of primitives, the DOM API ;) But the main reason is that I doubt template APIs provide any significant improvement over what current generation of libraries and frameworks have to deal with:

Now With Template APIs
Only some can be "rehydrated" from static HTML More or less assumes the template is already present as static HTML
Can create any complex structure on the fly Unless already defined as static templates, you will have to do what libraries and frameworks already do
Manually create DOM nodes and append/insert them into the document for dynamic parts such as lists/list items Manually create DOM nodes and append/insert them into the template (unless it's already a template, then manually instantiate and append/insert)
Manually keep track of changes to change the DOM only as needed Same (the proposal doesn't specify how the browser should work with data changes)
Manually track changes in lists of items (commonly using key attribute) to make sure lists are not re-rendered in their entirety Possibly same, possibly not. No idea how the browser will re-render items in a loop
Use any and all tools at your disposal: create any template/DSL you want, or use JS. Restricted to a very limited subset of moustache syntax with callbacks into JS. Conditionals left to libraries and frameworks, loops are not specified yet.

In my opinion, this is a marginal improvement at best, and the cost is significant. I've already talked abut it here: #704 (comment)

@bahrus
Copy link

bahrus commented May 5, 2019

Thanks @dmitriid for taking the time to patiently explain why I am wrong :-)

To be honest, I've come to realize that there seems to be an "East is East and West is West and never the twain shall meet." I think we are at opposite poles in terms of what appeals to us, and the point of discussing further seems to be pointless, though it makes for good comedy. Maybe we should start an act? :-)

For example:

Template instantiation presumes that the template already exists.

This is our fundamental disagreement? Is this a "chicken or the egg" type puzzle?

It exists.. And is widely used..

Why not make it even more useful?

somehow create the template (how?) ...
However, the only reasonable API for creating those templates? Probably .innerHTML :)

In the context of an HTML Module, the mystery eludes me. Why use an api? Why not type the text of the template into a static file? That doesn't use innerHTML. Or copy and paste from the first link above into a static file? I don't see an innerHTML button on my mouse. If the template to be cloned contains some pieces of semi-live data (but which will be repeated in every instance on the client) why not use PHP? Asp.net? The sky is the limit. Few if any of them use innerHTML to my knowledge. Surely, you know this, so I start to think you are speaking in some deeper level way beyond my comprehension. Are you using "innerHTML" as an allegory for something deeper? :-)

I'm afraid I don't have a good ear for poetry :-)

I get that I'm the freak here who likes the syntax of Vue and Svelte more than JSX. (Actually, I'm pleasantly surprised you appear to be warming up to template literals in your JSX example?) I work with JSX on a daily basis, and isn't my cup of tea. Sorry. That's just me. There's no accounting for taste.

And we still have plenty of performance problems, which has certainly not increased my affinity for it.

And by the way, I do share the concerns raised by Jan and Mildred above that the ability to apply transforms beyond what inline binding can provide is quite important.

If I may be so bold, could I suggest that what is really bothering you is that the proposal appears to favor the syntax of Vue and Svelte over your beloved React? I think that is a valid concern, but that may just be a kind of "illusion". In fact, I suspect it's that very concern that causes the proposal to be so vague and leave so much up to the frameworks, which you are using against it. I see that as a feature, not a bug.

I look for common goals when I communicate with others-- surely performance is a common goal? But you don't seem to want to engage in the question I find most interesting -- whether it's faster to generate a list by:

  1. Cloning a template with placeholders for data.
  2. Filling in the data into placeholders (which could already be quite fast, but template instantiation would make faster), and
  3. Appending to the container using appendChild,

vs.

...
Whatever it is that your compiler does with the JSX / template literal you provided?

And that's the thing. The compiler would presumably do whatever is fastest. That could, today, already be using templates without the instantiation proposal. Or maybe not. Maybe it's lots of appendChilds, which I think React uses. Maybe its lots of innerHTML's, which I think React used to use, but concluded lots of appendChilds is faster (I remember reading that a long time ago, but can't find it now). Interestingly, the folks at ag-grid concluded the opposite (see Hack 5). So I don't know what to think.

I'll give you the last word as we are going in circles

@dmitriid
Copy link

dmitriid commented May 5, 2019

@bahrus I engage in this conversation in good faith. You engage in thinly veiled trolling.

For the sake of others in this thread, I'll respond to some of the things you mention:

In the context of an HTML Module, the mystery eludes me. Why use an api? Why not type the text of the template into a static file?

Because not everything is a static HTML file. For a person who "deals with JSX daily" this should be obvious.

If the template to be cloned contains some pieces of semi-live data (but which will be repeated in every instance on the client) why not use PHP?

Because not everything is a static HTML file. Because not everything is HTML generated on the server and sent to the browser.

I get that I'm the freak here who likes the syntax of Vue and Svelte more than JSX.
If I may be so bold, could I suggest that what is really bothering you is that the proposal appears to favor the syntax of Vue and Svelte over your beloved React?

  1. React is not my beloved.
  2. Vue's template syntax is completely different from chosen in this proposal
  3. Svelte's template syntax is completely different from chosen in this proposal
  4. The proposal specifically states which syntax has been chosen

One more reason to think that from the offset you weren't interested in an honest discussion.

I suspect it's that very concern that causes the proposal to be so vague and leave so much up to the frameworks

The proposal is very concrete in all the places that sort of really matter (how to get data into the templates), and is vague in all the places where it matters (conditionals and loops).

So, no, I'm not thrilled by things being left to library and framework authors yet again when we could have high performing APIs built into the browsers, and not reimplemented time and again in incompatible ways in hundreds of different frameworks.

But you don't seem to want to engage in the question I find most interesting -- whether it's faster to generate a list by

Your bullet points are invalid in the context of the proposal. Moreover, I gave a concrete, simple example, which would be rather painful to implement within the confines of the proposal, especially when HTML is generated dynamically.

I also showed all the ways the proposal is not strictly or much better than existing approaches for all the reasons that the proposal omits, and for all the reasons that this entire discussion deliberately omits.

If you think this proposal obviates the need for existing frameworks, or that React/Vue/Preact/morphdom/hyperHTML/ will give up on their declarative APIs on top of their internal Virtual DOM implementations, your wishful thinking is much greater than mine. This proposal can't even tell how to deal with lists of DOM elements, or how a browser should behave when the actual .update call happens.

So I don't know what to think.

Indeed.

@bahrus
Copy link

bahrus commented May 5, 2019

Apologies, @dmitriid, for my overly defensive assumption about your preference for JSX over Vue / Svelte style syntax. I'm sure it was me reading too much, I guess, into your use of the word "moustache" in what I thought was a critical paragraph, combined with the previous comment, who said (I think) that he agrees with you, that he too finds the moustache syntax problematic in the same sentence, and which you didn't refute.

I'm sure I wasn't very clear, but what I was trying to say was the opposite -- that the proposal is designed to work with frameworks (I think) or additional helper libraries, not to obviate them.

I will work on my communication skills in the future to figure out how I can engage in these discussions, and not come off as trolling, as I seem to be misunderstanding everyone.

Apologies again.

@Jamesernator
Copy link

@dmitriid While the precise syntax and semantics of what builtin templates will be included are not worked out yet ((see this issue)). But it'll almost certainly look something like this:

<div>
  <template directive="if" value="loading">
    Loading...
  </template>
  <template directive="else">
    <template directive="for-each" in="items">
      <div>{{i}}</div>
    </template>
  </template>
</div>

Regardless of what the builtin semantics are I don't see how it really matters, the point of the default template instantiation processor is not to provide a turing complete language capable of doing any computation you could imagine but rather provide a good subset for a good chunk of typical use cases.

There's always gonna be use-cases that template instantiation is not going to solve by itself, for those cases there exists the DOM to do whatever manipulation on whatever data-structures you might need and quite importantly TemplateParts with which to have your own insertion points for data from those structures.


It's also not really clear to me what you're arguing, you keep referencing your original post as if virtual DOM is some holy grail of performance and usability which is a highly subjective point at best. I personally prefer the declarative HTML form as it's a lot easier in a large amount of structural elements to separate your presentation from any data you carry.

Regarding your points:

somehow create the template (how?)
somehow insert it into the document or into another template (how?)
when values change, call .update, and the values will be updated (the proposal)

Yes, I'm not clear what is wrong with this? There's already perfectly canonical ways of inserting it into the Document/ShadowRoot e.g. .appendChild. The typical flow would look like this:

  • get the template from the document or an HTML module
    • e.g. import { loadingTemplate } from "./loadingTemplate.html"
  • instantiate it with a processor (which might be the default processor)
    • e.g. const instance = loadingTemplate.createInstance()
  • append it to whatever DOM you want
    • e.g. shadowRoot.appendChild(instance)
  • update it in response to data changes
    • instance.update({ changedKey: newValue })

you can very easily wrap this flow into a PureComponent helper or something in a lot of cases. For other cases you can always use DOM as if you don't know ahead of time what structure you might be constructing you probably need some imperative logic anyway.

Because not everything is a static HTML file. For a person who "deals with JSX daily" this should be obvious.
Because not everything is a static HTML file. Because not everything is HTML generated on the server and sent to the browser.

Heaps of stuff is just static content that you want to include in the page for its structure rather than anything dynamic. The whole point of template instantiation is to also provide you insertion points where you want dynamic data to go. And as I've already said, even when things are dynamic you almost always have a certain number of insertion points.


One final thing, the spec text in this repository is not necessarily up to date or what will be implemented in browsers. There's currently a WIP implementation in Chrome to implement template instantiation which simply by looking at the WebIDL definitions I can see has a fair few differences from the spec in this repository.

@dmitriid
Copy link

dmitriid commented May 6, 2019

I would like to remind you all of the original question raised:

Before adding anything as complicated as https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md it is better to ensure it improves the platform.

Once again, this proposal introduces an objectively complex:

  • a new templating language (incompatible in syntax and behaviour with everything else in the ecosystem)
  • that is both very limited in functionality (poorly specified/underspecified if and foreach as the only control structures) and unlimited in functionality (we can call functions in the host language)
  • that is alien to the system (literally nothing in the platform looks or behaves like this)
    that requires a different parser, a different execution model, hooks into the rest of the platform to make it workable
  • solves the main use case of composing components poorly, or not at all (you need to nest template elements within template elements within template elements...)
  • doesn't specify how you would instantiate elements not from static sources, but from dynamic data (I'm guessing .innerHtml again?)

And introduces only marginal improvements (if they are improvements at all). And on top of all that it basically leaves nearly everything to the libraries and frameworks.


Yesterday I came up with the simple "Loading..."/list-of-DOM-elements example. The solution suggested by @Jamesernator already shows how horribly cumbersome the alternative provided by proposal is.

Then I realised there's an another example:

<button onclick="{{ f }}"></button>

The proposal provides no way of dealing with this except manually parsing this, figuring out what the attribute is, setting up the listeners etc. That is, once again, delegating everything to the libraries and frameworks. And these are not corner cases, or edge cases. These are actual use cases that people come up against every single day.

So, instead of giving the power to end devs, once again, the standards opt for "we're going to introduce a very limited, underspecified, cumbersome API that is only aimed at developers of libraries and frameworks". Can we stop this trend, please? :)


@Jamesernator

It's also not really clear to me what you're arguing, you keep referencing your original post as if virtual DOM is some holy grail of performance and usability which is a highly subjective point at best.

Would you please give me at least a sliver of respect and read what I am actually writing. For example, here: #704 (comment) where I actually talk about killing the need for VDOM

@dmitriid
Copy link

dmitriid commented May 6, 2019

Ah. I clean forgot one more thing. This proposal lacks a simple cost-benefit analysis.

The proposal lists 9 use cases and provides satisfactory solutions to at most one or two if them.

And despite this fact, and despite the fact there’s an ongoing debate around implementation of conditionals and loops, etc. etc. of course Chrome is already implementing it.

Because who wouldn’t want another half-baked underspecified vague standard in the browser around which everyone will be busy building new workarounds and abstractions (because its nigh unusable otherwise).

@Jamesernator
Copy link

Jamesernator commented May 7, 2019

Would you please give me at least a sliver of respect and read what I am actually writing. For example, here: #704 (comment) where I actually talk about killing the need for VDOM

I have read it and your proposed solution is essentially still just a VDOM/tree-diff except generating a patch on the server-side instead of the client. However this forces the server to handle all work which may or may not be desirable depending on your application.

It's definitely not bad but it definitely has some major drawbacks:

  • Behaviour can't be shared between different server languages for public components so now you wind up re-implementing things in every other language
  • You require a persistent connection to a server and can't delegate work easily to the client side
  • Offers no solution for structure on components which need to be rendered on the client side due to client-side information required

And also note this approach still can suffer the usual tree-diff issues:

  • Elements might lose state if they're removed and re-rendered (e.g. <video>), in contrast with templates you have the decision as to whether or not to re-use the same tree again (or any strategy you want)
  • You need another copy of the DOM itself rather than just the data that changes. In this case it lives on the server side and costs server resources to maintain

already shows how horribly cumbersome the alternative provided by proposal is.

"horribly cumbersome" is highly subjective, regardless you could propose your own alternative syntax if you feel strongly about it and believe that more concise syntax would significantly improve adoption.

I would personally prefer something like <for.each in="loading"><!-- contents --></for.each> but it's not clear what impact this would have on existing HTML parsers that expect only <template> to have parsed but inert content. I think I'll raise the matter on the issue about the default processor because I'm not super clear if it would actually be a problem.

Then I realised there's an another example:

Again there's an open issue which is specifically about the capabilities of the default processor. I do suggest proposing any things you think will be really common use cases there.

The proposal provides no way of dealing with this except manually parsing this

Even in underspecified current spec there's absolutely nothing that requires you to parse it yourself. Just delegate to the default processor to handle parsing and replace any AttributeTemplatePart with your own parts.

we're going to introduce a very limited, underspecified

I highly doubt that proper spec text won't be released before the feature is shipped.


Regarding your original points:

a new templating language (incompatible in syntax and behaviour with everything else in the ecosystem)

I don't know what you mean by "incompatible in syntax and behaviour", there's a wide difference in syntax and behaviour (for example Vue uses v-bind:attr="someProp" whereas JSX uses attr={someExpression}). What do you consider to be "compatible in syntax and behaviour"?

The proposal has to choose some syntax it can't possibly choose all of them, I don't see minor syntactic differences being the deal-breaker of the proposal.

that is both very limited in functionality (poorly specified/underspecified if and foreach as the only control structures) and unlimited in functionality (we can call functions in the host language)

This doesn't seem to be so much an issue with template instantiation itself as a lack of concrete spec text.

that is alien to the system (literally nothing in the platform looks or behaves like this)

True, but that's true of any new feature.

that requires a different parser

Nope it doesn't, I've implemented an almost full polyfill of it and the way it works in both the current spec and my polyfill is it simply looks at attribute and text nodes for {{. For example consider <p attr="{{value}}">, we don't need a new parser to handle this at all, we just check if the attr contains {{ and if it does we replace it with a Template Part.

a different execution model

I don't understand this you'll need to elaborate. But it seems to me that anything would be a "different execution model".

hooks into the rest of the platform to make it workable

Again I don't understand this. It seems to me that anything would have "hooks into the rest of the platform", that's kinda the point of Template Parts is that they hook into the DOM and can dynamically update minimal parts of the DOM.

solves the main use case of composing components poorly, or not at all (you need to nest template elements within template elements within template elements...)

This seems to be quite syntax opinionated but you haven't actually proposed a syntax alternative that you'd prefer.

doesn't specify how you would instantiate elements not from static sources, but from dynamic data (I'm guessing .innerHtml again?)

You shouldn't need .innerHTML for anything. Either use DOM for the cases when you actually truly arbitrary structures, but templates should suffice for anything with any degree of structure (which is most things).

@dmitriid
Copy link

dmitriid commented May 7, 2019

@Jamesernator

I have read it and your proposed solution is essentially still just a VDOM/tree-diff except generating a patch on the server-side instead of the client.

There’s more there than just server-side rendering, but I’ve removed the rest if my comment, as it was written in a provocative tone. I’ll come back with a proper detailed answer that also keeps the original question in mind.

I apologize to thise who had to read my original reply.

@mercmobily
Copy link

This has been quiet for nearly a year... is template instantiation still a "thing"? What's the current status of it? (As in, what should happen at this stage to see it progress?) I can't find references to it anywhere other than the proposal itself and a few articles from 2017 pointing to it.

@rniwa
Copy link
Collaborator

rniwa commented Feb 11, 2020

This has been quiet for nearly a year... is template instantiation still a "thing"? What's the current status of it? (As in, what should happen at this stage to see it progress?) I can't find references to it anywhere other than the proposal itself and a few articles from 2017 pointing to it.

Yes, we're still interested in pursuing this proposal & solutions in this space. However, we're currently focused on resolving gaps in the existing Shadow DOM & Custom Elements APIs like improving the focus & accessibility support.

@rniwa
Copy link
Collaborator

rniwa commented Oct 27, 2020

I've worked with @hober @justinfagnani @yuzhe-han @mfreed7 to make a refined API proposal for templates:
https://github.com/rniwa/webcomponents/blob/add-dom-parts-proposal/proposals/DOM-Parts.md

There is a PR to merge it into this repository.

We'd discuss it at tomorrow's TPAC breakout session.

@trusktr
Copy link
Contributor

trusktr commented Jul 25, 2022

It seems like this spec will have to lead to browsers allowing native templating to be faster and more space efficient than frameworks in benchmarks (f.e. see js-framework-benchmark) or we'll end with something that people won't use; they'll just use the frameworks (which already have good DX, f.e. JSX with full type support in TypeScript, along with compile-time optimization thanks to new tools like Solid.js).

A solution would need to be as easy, or easier than these frameworks for it to be really adoptable. If people are going to end up writing frameworks on top of DOM Parts (or related APIs), then end users will ultimately be using frameworks, and will end up still choosing whatever works best (whether built in or not).

Has this been given consideration?

@sashafirsov
Copy link

When it goes to speed of template rendering, think of

  • rendering while data still load
  • rendering while the template is still in progress of load.
    I.e. both could (and for performance reasons should) be available via streaming and result available as stream as well.

To be efficient the template could be tuned against data schema. Which would make generated code compiled up to binary level and processed in parallel threads.

Which goes back to beautiful concept of XML+XSLT 😛

Now tell me that reasons above are not more significant than discussion ^^. Replace the XML/XPath/XSLT with equivalents supported by browser and you have #1 by performance and memory efficiency. From another hand what prevents to use XSLT as it is already part of browser stack?

  • indirect JS data access, no streaming of JS-data
  • syntax( with so many transformers around, we could change it at no time )
  • lack of modular development ( same - transformer would solve it )
  • no DOM binding/rendering directly into DOM ( same reason as discussed diffs for VDom ^^)

The polyfill could go only to certain level of performance for such design. Most heavy lifting would be on browser engine. But even streaming render in polyfill would beat React and similar VDOM rendering engines.

@LucaColonnello
Copy link

LucaColonnello commented Jul 26, 2022

@sashafirsov in terms of what this is trying to achieve, sure, XML+XSLT seems a good comparison. However there’s a reason those template systems have been suppressed in the past for web technology and data binding in the first place.

They are very limited in developer experience and are in no way close to or as powerful as what we do in modern web development.

Writing an if statement in a string attribute was never an appealing approach, and still isn’t.

What I would love to see as an alternative, is a low level DOM API that give a template and all the interpolations with data, provides hooks to apply changes in places. So the API does not dictate how the changes should be performed and what syntax or evaluation is best for you, but it simply analyses at compile time the interpolations.

The browser is already able to parse HTML strings in an efficient way, so it could go a step further and tell us “given template x with all the interpolations (syntax to be defined), these are the dynamic parts, the ones that are bound to change due to interpolations”.

I feel like this could be a good middle ground as frameworks can provide whatever DX they want, leaving the low level mechanism to browsers.

This can be done at compile time for every template, it supports SSR and it’s easy enough for browsers to implement.

It also gives us something lime Solid or Svelte but at runtime (compile runtime actually), rather than having to use a compiler.

I think the DOM Parts proposal comes very close to this, and frameworks like Svelte and Solid have proven that this is a good direction. We probably just need something, which could be template based or not, to create the DOM parts given an html.

I think this proposal goes too far as it’s very restrictive to mustache…

@bahrus
Copy link

bahrus commented Jul 26, 2022

@LucaColonnello

Writing an if statement in a string attribute was never an appealing approach, and still isn’t.

Never appealing to whom? If just you, with all due respect, I don't think that should drive this decision :-)

Vue, AngularJS, and other templating syntaxes use admirably concise if/else logic based exclusively on attributes, that appeals to many members of the development community, for its sheer conciseness and readability, I think. We should be so lucky if template instantiation could support that. Unfortunately, due to Template Instantiation wanting to best represent how the conditional is understood internally, without a build step, it may not be practical / worth the effort to represent conditionals so concisely, and may have to mirror the verbosity of svelte, React, and tagged template literals. This is what that looks like in practice I think we can live with that.

A number of JSX-based frameworks, including Solid JS also don't consider use of attributes to be so scandalous when representing conditional statements. I just don't see what is so unacceptable about use of attributes, especially if it can make the syntax less verbose. What am I missing?

It should be remembered that although querySelectorAll barely made a dent in JQuery's market share, JQuery was able to reduce its size and complexity by utilizing it (it had to keep the sizzle engine in place because it supports queries not yet supported by querySelectorAll), while preserving its API. I think querySelectorAll was worth it for that reason alone. I don't know what percent of the existing frameworks would lean on Template Instantiation in a similar way. Certainly not 100%, but I suspect a good number would. And it could also lower the barrier to entry of new ideas, especially if template instantiation's performance is in the same ballpark as existing ones. I agree it might be a hard sell for libraries that clock in under 3KB, if the performance is the same. But even for those libraries, hydrating with declarative shadow DOM, from a shared template, is a prominent use case that I suspect even those libraries would make use of.

What I think really underlies the concern that is being raised here is whether unfettered access to the JavaScript runtime should be permitted in the template syntax (however, apologies if I am misreading the concern).

If there is a goal to reduce the need for UI libraries on top of Template Instantiation, then that is certainly a factor to consider, and I agree that without it, the number of use cases of relying exclusively on Template Instantiation is reduced. But I don't think it should be the primary goal (and has been clearly stated as not being the goal).

Maybe there can be multiple levels of access permitted, passed in as an optional parameter to the template instantiation engine, based on the trust level that is in place between the code that executes the template instantiation, and the providers of the templates (similar to how iframes use the sandbox attribute). The parameter could have at least three levels:

  1. Only built-in, minimal if/else / looping logic with only safe, low hanging fruit for common things like null condition alternatives (??) number / date formatting, etc. This syntax could be used by CMS's like WIX or Micro Front Ends.
  2. Allow unfettered access but only within a provided Shadow Realm.
  3. Allow 100% unfettered access.

Microsoft did something like that with XSLT, actually (allowing JavaScript execution), as shown here. That ability was removed from .net core, unfortunately, if anyone cares.

@dmitriid
Copy link

dmitriid commented Jul 27, 2022

@bahrus

querySelectorAll is a good example of how the designers of that API looked at a useful practical ergonomic userland API and butchered its implementation in the browser.

jQuery:

  • a single API that returns an array of elements, or an empty array of no elements are found

Browsers:

  • two separate APIs. One to retrieve a single element. One to retrieve multiple. Both throw exceptions

  • querySelectorAll doesn't return an array. It returns a NodeList that didn't even have a convenience method like forEach for two or three years after querySelectorAll was introduced. And you still need to call Array.from on it to have access to anything useful like map or reduce.

I wonder why it didn't make a dent in jQuery. Perhaps because it objectively sucks and you still need to write otherwise useless wrappers to make it practical?

What is it with browser implementors that they look at powerful, practical userland abstractions and go: nah, we're going to implement none of that. We're going to implement primitive half-assed solutions that still require as much, or more, code to work with, let libs and frameworks deal with this and call it a day.

@trusktr
Copy link
Contributor

trusktr commented Aug 17, 2022

Writing an if statement in a string attribute was never an appealing approach, and still isn’t.

Sounds similar to Vue (which adopted syntax patterns from Angular) and other systems with similar HTML-first syntax. And quite a number of people like Vue.

I like HTML-first languages too, but I can't live without the complete type support and intellisense that JSX+TypeScript offers out of the box.

I just looked at XSLT for the first time. I does remind me of Vue & Co in some ways. It also reminds me of Alpine.js in some ways.

It would be as if we took Alpine.js or a similar lib, and instead of basing everything on attributes, had a more custom element approach (f.e. <for-each>, <value-of>, etc).

I believe the attribute approach is better, because I feel that custom elements are for rendering and content, not for logic. It can get tricky when such elements get in the way of CSS selectors. For example a <value-of> element introduced into markup would have effect on nth-child selectors and inadvertently break them.

I believe we need either attribute syntax, or new syntax, to avoid changing the structure of our content.

Attribute syntax is perhaps on the edge of that line (the other side being elements for logic) because it does have effect on selectors, queries, and content structure.

New syntax could entirely avoid the issue (f.e. handlebars syntax for sake of example).

So the API does not dictate how the changes should be performed and what syntax or evaluation is best for you, but it simply analyses at compile time the interpolations.

This still raises the concern: if this API is only a very low level thing, then people will make frameworks on top of it. If those new frameworks are not faster than those pushing the current boundaries like those in js-framework-benchmark and other benchmarks, then people will just use the other frameworks (with added benefits that they also work serverside without DOM APIs).

Furthermore, if a Parts API or similar becomes simply a faster way to manipulate DOM, the frameworks like Solid.js/React/Vue/etc will simply use it under the hood for that purpose, while making their code even less understandable in the same way that frameworks have adopted document.cloneNode() as a faster but less idiomatic way of creating DOM than document.createElement().

So if Parts allows more speed, frameworks are gonna start writing Parts code to win in benchmarks, leading to probably insignificant gains for most people.

Back in the day, Backbone.js+Handlebars was perfectly fine for most use cases as far as performance was concerned; we accomplished most things back in the day on slower hardware using those tools, and people were ok, while appropriate escape hatches were used when needed.

React got faster, but the real win was the developer experience improvement.

If Parts is only going to speed things up a small amount, but require more complicated code, leading to only framework authors using it, then it doesn't seem so valuable with the cost of its API surface area.


If this is going to work, IMO, it needs to provide a really good DX (dev experience) that can compete with the status quo.

In its current state, if frameworks will merely use it for small speed gains, it will be quite a lot of engineering effort for almost nothing.

In its current state, if it will not provide any perf gain over cloneNode + setAttribute + set JS props approaches (noting that it currently does not provide a better DX), then it may end up as wasted engineering effort.


Keep in mind that we can write dependency-tracking reactivity systems like those that Solid.js, MobX, Knockout, and Vue have, in 85 lines of code.

I can pull that out of my back pocket in any app and have a great developer experience for manipulating DOM:

import {createSignal, ccreateEffect} from './file-with-85-loc'

const el = document.cloneNode(template)
const countSpan = el.querySelector('span[\\:text=count]') // f.e. <span :text="count"></span>
const count = createSignal(0)

setInterval(() => count.set(count.get() + 1), 1000) // increment every second

// This block of code reruns any time `count` changes
createEffect(() => {
  countSpan.textContent = count.get()
})

// Now the DOM updates every second.

This dependency-tracking reactive pattern is cleaner, and greatly improves simplicity of code bases (I haven't covered that here, but an article is coming soon).

So what does Parts really achieve that will cause people to really want to use it?

I feel like this could be a good middle ground as frameworks can provide whatever DX they want, leaving the low level mechanism to browsers.

They indeed could! But there would need to be a benefit. Better performance for end users?

For example, why would Solid.js adopt it under the hood?

Solid.js already compiles JSX templates to the same type of code as I wrote in the previous example, plus it provides a runtime html template string tag that compiles html syntax with interpolations (same syntax as lit-html essentially) to the same sort of above code output at runtime for projects without build tools.

The DX these frameworks have are already good. So there'd need to be a strong reason to adopt it.

What is that reason?

it supports SSR and it’s easy enough for browsers to implement

Would this require a DOM polyfill in Node.js? DOM polyfills (jsdom, undom, etc) have been proven to be slow compared to the fastest frameworks in benchmarks (by considerable margins) making the use of string interpolation/concatenation.

The slow speed of fake DOM in Node.js could be another reason that libs will avoid the Parts API for SSR, and hence will support their own system rather than multiple.

It also gives us something like Solid or Svelte but at runtime (compile runtime actually), rather than having to use a compiler.

Check out Solid's html template tag, using no build tools, on CodePen. The code:

<style>
	html, body { font-size: 1.5rem; }
</style>

<script type="module">
	import {createSignal, createEffect} from 'https://cdn.skypack.dev/[email protected]';
	import {render} from 'https://cdn.skypack.dev/[email protected]/web';
	import html from 'https://cdn.skypack.dev/[email protected]/html';

	const [count, setCount] = createSignal(0)

	setInterval(() => setCount(count() + 1), 1000)

	const div = html`
		<div>
			<strong>Hello</strong>&nbsp;
			<em>you!</em>

			<p>The count is: ${count}</p>
		</div>
	`

	console.log(div instanceof HTMLDivElement) // true

	document.body.append(div)
</script>

What is the Parts API going to improve in that demo, for example?

@trusktr
Copy link
Contributor

trusktr commented Aug 28, 2022

What is it with browser implementors that they look at powerful, practical userland abstractions and go: nah, we're going to implement none of that. We're going to implement primitive half-assed solutions that still require as much, or more, code to work with, let libs and frameworks deal with this and call it a day.

Insulting words aside, I do feel your sentiment: some APIs really are too complicated. MutationObserver is an example.

Please let's not get rude towards people, but only express what we think are better solutions.

@sashafirsov
Copy link

sashafirsov commented Aug 28, 2022

let's not get rude
hmm, when addressed towards people it is not just insulting but also counterproductive. But towards API implementation it is important to look into depth and get over 5 whys on why there is a frustration answer would reside on either decision process or particular API developers vision or lack of.

Many stones been thrown forwards NodeList and enumerable in JS, Mutation observer, etc. Will be more towards templates.
reading over main goal statement, it look a right direction.

  1. Provide a high performance primitive for template engines & frameworks to build on top of.
  2. Provide a minimal high-level template support so that authors don't necessarily rely on a library or a framework to make a good use of templates
  3. Provide a common framework by which different template engines & frameworks can work together

But once API would select limitations over openness, closed rather evolutionary approach the fate would follow those 😞 samples.

JS centric vs declarative HTML approach

While JS seems to allow the max flexibility, it is a source of mistrust and no-js declarative web app is winning more popularity.
By providing declarative-first implementation you will gain

  • of course trust from all kind of consumers. Penetrate declaratively programmed web app would be almost impossible.
  • performance the browser engine is empowered for lots of optimizations as would not need to account the JS driven app logic. From parallel rendering to on-demand hydration.
  • reliable base for API which would match the declarative stack. When there is a need for JS trickery, it would less interfere with browser engine based on declarative HTML treatment.

the template source and need to inject template into DOM

Not sure why this discussion is missing the whole ecosystem of HTML modules, externally defined templates and (external-)document with multiple templates as representation of library concept.

Good solution would account those most used aspect.
My take the templates shall be part of module loader and be a application-wide (domain or descriptor-defined) cashed as document parts. As such template DOM/root used in current document. But for externally defined template source document the templates have to be reused without the need to be injected directly into current document. Page changes, templates remain in memory and ready to be reused or offloaded. Which of course is not (yet?) a part of current DOM model.

Back to subject of current discussion, how the template is exposed to JS would be up to developers, but platform should support all essential needs:

  • available from same document DOM when provided
  • load from external document (DOM), loaded on demand
  • from html module ( ideally would match case ^^ but seems proposal goes "wrong" direction towards string presentation)
  • re-defined in runtime(dynamic templating) by innerHTML/custom DOM fragment.

may be few more methods.

@dmitriid
Copy link

@trusktr

To quote Figma

the web wasn’t designed as a general-purpose computing platform. It started off as a technology primarily for documents and had a whole bunch of stuff for application development bolted on top. That stuff usually took the form of specific APIs for one-off cases instead of providing general primitives that can be used to implement all sorts of things.

To quote Dan Abramov of Redux and React fame

React users would love to not have to npm install a date picker and bloat their bundles! If they need to "use the platform" then why doesn't that platform ship the features they actually ask for? Instead of a <carousel> they get an <aside>. Features like service workers are touted as a solution to many problems in the web, but their ergonomics are so under-designed that people actually have to change domains to bust the cache from a broken build (I’m not making this up).


We're now in a sitation when web browsers are hellbent on removing alert/prompt/confirm withouy providing any usable replacements. Well, they've now rushed <dialog>. Which they argued should propbably be removed from the spec because no one wants to implement it due to a list of issues that no one has solved since


The whole web component that web browsers have sunk hundreds of millions of dollars into has us stranded with:

  • cumbersome neither-low-nor-high-level API that even Google's own projects cannot use (AMP switched to using Preact to author them because the API is not conductive to development)
  • "this is just html" components that have to have more and more Javascript bolted onto them to fix things like form participation, and in the future acceccibility issues
  • are now poisoning all other standards because they now have to make sure those standards play nice with WCs and Shadow DOM (e.g. Scoped CSS which covers ~80% of use cases people use WCs for is now being butchered to accomodate Shadow DOM because of course it is)
  • cannot be used with SVGs (that is, WCs cannot appear inside SVGs, see twit-sized explanation)

It looks like literally zero long-term planning for most of the features just dumped into the platform. None whatsoever.

You want developers like me to stop being "insulting"? Then stop adding things haphazardly with no forethought or planning. As QuirksMode suggested in 2015

Why should we push the web forward? And forward to what, exactly? Do we want the web to be at whatever we push it forward to? You never hear those questions.

The innovation machine is running at full speed in the wrong direction. We need a break. We need an opportunity to learn to the features we already have responsibly — without tools! Also, we need the time for a fundamental conversation about where we want to push the web forward to. A year-long moratorium on new features would buy us that time.

Instead we get * vaguely gestures at everything *.

@sashafirsov
Copy link

The question of long-term vision and high level goals is not something folks are not thinking those days. The sad side not in WICG forum which meant to be incubator for innovations.

WCCG is asking for Where do you see yourself in 5 years? , I am collecting thoughts myself 1, 2, but still looking into proper forum to discuss the high-level goals and creation of non-conflicting concepts and implementations.

Aforementioned is just a sample how some folks see the vision as important part of current decision making. Would be nice to see such vision fit here, in templates implementation discussion.

My take: templates are essential part of web application development SDLC. Which includes modularity, concept of libraries with own scope, ability to load as a bulk, caching across page loads, etc.

In the ongoing web app stack templates are essential part of declarative custom elements; IMO outdated concept of declarative shadow DOM. The parts compete with slots and meant to be just a syntax sugar over what slots meant to support out of the box( needs ATTRIBUTE and PART be exposed as HTML tags ).

In order to have a clear picture the concepts above at least meant to be discussed as a whole.

@dmitriid
Copy link

@sashafirsov

5 years is not nearly long enough. The ongoing multi-hundred-million-dollar saga of web components is 11 years this year, with no end in sight. And we've yet to see a coherent answer to what the end goal is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests