Question: Webpacker integration -> using local manifest.json for server-side rendering #739

randyv12 · 2017-06-09T15:02:50Z

Hello!

I'm using the most recent version of react-rails (2.2.0) and Webpacker (1.2). One of the issues I ran into this week was server-side rendering, not fetching assets locally, but instead fetches from the remote asset host. I'm curious if this is fixed, if it's a bug (first of all), or if it will be-supported in the future or which gem will support it?

Explanation:

If Webpack detects that there is an asset_host set in a config, it will prepend the asset_host url in the manifest.json file in Webpacker (1.2) when it precompiles assets via rake webpacker:compile

webpacker/lib/install/config/webpack/configuration.js

const ifHasCDN = env.ASSET_HOST !== undefined && env.NODE_ENV === 'production'
const devServerUrl = `http://${devServer.host}:${devServer.port}/${paths.entry}/`
const publicUrl = ifHasCDN ? `${env.ASSET_HOST}/${paths.entry}/` : `/${paths.entry}/`
const publicPath = env.NODE_ENV !== 'production' ? devServerUrl : publicUrl

So when we precompile assets, our manifest.json file will contain, https://asset.host.com
and It will look like this

{file.js: 'https://assets.com/assets/file.js}

or like this

{file.js: '//assets.com/assets/file.js}

So when we server-side render using react ujs, using the web packer manifest container. The container will read from the local manifest.json with the asset host in it and recognize the asset_path that starts with 'http'. And when it does that, it will fetch the assets remotely, when server-side rendering. We don't want that, I think, we should just fetch it locally (since we have the files already)

react-rails/lib/react/server_rendering/webpack_manifest_container.rb

      def find_asset(logical_path)
        # raises if not found
        asset_path = Webpacker::Manifest.lookup(logical_path).to_s
        if asset_path.start_with?("http")
          # Get a file from the webpack-dev-server
          dev_server_asset = open(asset_path).read
          # Remove `webpack-dev-server/client/index.js` code which causes ExecJS to 💥
          dev_server_asset.sub!(CLIENT_REQUIRE, '//\0')
          dev_server_asset
        else
          # Read the already-compiled pack:
          full_path = Webpacker::Manifest.lookup_path(logical_path).to_s
          File.read(full_path)
        end
      end

Current Solution

To get around that and solve this issue we will have to create a separate manifest.json file (from Webpack, not Webpacker), which will not include the asset host, so we can locally reference them, without fetching assets remotely.

That, we can do on the Webpacker gem side and add in a separate webpack manifest plugin entry that creates a separate manifest.json for server-side rendering in a separate folder, something like:

webpacker/lib/install/config/webpack/shared.js

  plugins: [
    new webpack.EnvironmentPlugin(JSON.parse(JSON.stringify(env))),
    new ExtractTextPlugin(env.NODE_ENV === 'production' ? '[name]-[hash].css' : '[name].css'),
    new ManifestPlugin({ fileName: paths.manifest, publicPath, writeToFileEmit: true }),
    new ManifestPlugin({
      fileName: '../pre-render/manifest.json',
      publicPath: `${paths.prerender_path}/`,
      writeToFileEmit: true})
  ],

Then, after that, we will need to read that server-side manifest json file in
react-rails/lib/react/server_rendering/webpack_manifest_container.rb
so we can use the local assets that are precompiled, for server-side rendering

something like this:

module React
  module ServerRendering
    class WebpackerManifestContainer
      def find_asset(logical_path)
        path = ::Rails.root.join(File.join(Webpacker::Configuration.output_path, '/pre-render/manifest.json'))
        full_path = ::Rails.root.join(File.join(Webpacker::Configuration.output_path, JSON.parse(File.read(path))[logical_path.to_s]))

        if full_path
          return File.read(full_path)
        end

       // else do original find_asset stuff
    end
  end
end

The text was updated successfully, but these errors were encountered:

justin808 · 2017-06-13T06:55:21Z

This discussion might be related, as Webpacker currently wants hosts in the manifest.json: rails/webpacker#464. I'm trying to get this changed so that React on Rails might depend on Webpacker and not Webpacker Lite.

randyv12 · 2017-06-14T00:17:12Z

thanks, we can close this now, i'll just look forward to the changes and keep a close eye on the repo.
i found this earlier today, this is a gist that creates a separate config for client side and server side
https://gist.github.com/jslatts/f9dec699e8fd853590f520f62d5bf21a

justin808 · 2017-07-26T04:05:15Z

In case anybody is curious, I'm working on this right now. Stay tuned.

joeyparis · 2017-11-03T17:46:02Z

Do we have any updates on this? This is a relatively recent problem for me that didn't affect me for the last couple of months (since ~August) but suddenly I am having the exact problem described in this issue.

BookOfGreg · 2017-11-03T18:51:31Z

I'm glad to say React-Rails supports this for Webpacker 3! 🙌

$ ASSET_HOST=https://example.com rails c
Loading development environment (Rails 5.1.4)
irb(main):001:0> helper.javascript_pack_tag 'application.js'
=> "<script src=\"//assets.example.com/packs/application-ddc460fb88ce313d87be.js\"></script>"
irb(main):002:0> Webpacker.manifest.lookup('application.js')
=> "/packs/application-ddc460fb88ce313d87be.js"

Thanks for having me check @joeyparis, looks like we fixed it when we added Webpacker 3 support.
BookOfGreg/react-rails-example-app#5

What issue are you having?

joeyparis · 2017-11-03T21:09:33Z

@BookOfGreg,

When I am using webpacker's webpack-dev-server to watch individual files for compiling I get the following error when calling the react_component function in rails with prerendering set to true: bad URI(is not URI?): http://0.0.0.0:8080http://0.0.0.0:8080/packs/server_rendering.js .

I have traced the problem back to this line in the react-rails source code:

react-rails/lib/react/server_rendering/webpacker_manifest_container.rb

Line 45 in 16503ee

    
           dev_server_asset = open("#{ds.protocol}://#{ds.host_with_port}#{asset_path}").read

The problem is that the asset_path variable already includes the protocol and host_with_port values leading to the weird URI I get in the error above.

I've found that if I manually edit webpacker's public/packs/manifest.json file and remove the leading host from each attribute react-rails works as expected.

From this:

{
  "Digital.svg": "http://0.0.0.0:8080/packs/Digital.svg",
  "DirectMail.svg": "http://0.0.0.0:8080/packs/DirectMail.svg",
  "Email.svg": "http://0.0.0.0:8080/packs/Email.svg",
  "application.css": "http://0.0.0.0:8080/packs/application.css",
  "application.js": "http://0.0.0.0:8080/packs/application.js",
  "loading.gif": "http://0.0.0.0:8080/packs/loading.gif",
  "loading.svg": "http://0.0.0.0:8080/packs/loading.svg",
  "rocket.svg": "http://0.0.0.0:8080/packs/rocket.svg",
  "server_rendering.css": "http://0.0.0.0:8080/packs/server_rendering.css",
  "server_rendering.js": "http://0.0.0.0:8080/packs/server_rendering.js"
}

To this:

{
  "Digital.svg": "/packs/Digital.svg",
  "DirectMail.svg": "/packs/DirectMail.svg",
  "Email.svg": "/packs/Email.svg",
  "application.css": "/packs/application.css",
  "application.js": "/packs/application.js",
  "loading.gif": "/packs/loading.gif",
  "loading.svg": "/packs/loading.svg",
  "rocket.svg": "/packs/rocket.svg",
  "server_rendering.css": "/packs/server_rendering.css",
  "server_rendering.js": "/packs/server_rendering.js"
}

However, this requires manually editing the file each time I restart webpack-dev-server.

I am using webpacker 3.0.2 and react-rails 2.4.0.

Running your provided example returned the following with webpack-dev-server running:

$ ASSET_HOST=https://example.com rails c
[...]
Loading development environment (Rails 5.1.4)
2.3.1 :001 > helper.javascript_pack_tag 'application.js'
 => "<script src=\"http://0.0.0.0:8080/packs/application.js\"></script>"
2.3.1 :002 > Webpacker.manifest.lookup('application.js')
 => "http://0.0.0.0:8080/packs/application.js"

and the following without webpack-dev-server running:

$ ASSET_HOST=https://example.com rails c
[...]
Loading development environment (Rails 5.1.4)
2.3.1 :001 > helper.javascript_pack_tag 'application.js'
Compiling…
Compiled all packs in /Users/Joey/Sites/LeadJig/public/packs
 => "<script src=\"/packs/application.js\"></script>"
2.3.1 :002 > Webpacker.manifest.lookup('application.js')
 => "/packs/application.js"

BookOfGreg · 2017-11-03T21:25:26Z

Excellent @joeyparis, I didn't know it's specific to when the dev server is running. Now I can get about fixing this. I'll ping this chat when a fix is inbound 👍

joeyparis · 2017-11-03T21:27:42Z

Great to hear! Thanks for the quick reply.

Let me know if there's any more information I can provide. I'm happy to help you with debugging and testing in my environment if you need anything.

BookOfGreg · 2017-11-03T21:34:21Z

I'm pretty low on help in this gem at the moment and I'm currently working on .hydrate() #808 tonight.
I'd very much welcome a PR! My crude solution to this would likely be check asset_path contains // and only prepend #{ds.protocol}://#{ds.host_with_port} if not, then look at the neighbouring code to see if there are any more elegant options.

joeyparis · 2017-11-03T21:37:30Z

Let me see what I can do. I won't be working on this project again until Monday morning, but that's not to say I can't set aside some time this weekend to create a PR for the gem. I can certainly implement a crude check like that pretty quickly until a more elegant solution can be found. I'll keep you updated!

joeyparis · 2017-11-06T14:24:27Z

Didn't get a chance to tackle this issue this weekend, but going to get started on it now. Should have a PR here shortly.

Issue reactjs#739 In certain circumstances `webpacker-dev-server` includes the full path (protocol and host with port) in the paths generated for the packs/manifest.json file. This addition removes the protocol, host, and port from the asset path when present allowing assets to be correctly loaded.

joeyparis · 2017-11-06T15:19:22Z

I've created a crude but quick and easy solution!

# Line 41
def find_asset(logical_path)
  asset_path = Webpacker.manifest.lookup(logical_path).to_s
  if Webpacker.dev_server.running?
    ds = Webpacker.dev_server
    # Remove the protocol and host from the asset path. Sometimes webpacker includes this, sometimes it does not
    asset_path.slice!("#{ds.protocol}://#{ds.host_with_port}")
    dev_server_asset = open("#{ds.protocol}://#{ds.host_with_port}#{asset_path}").read
    dev_server_asset.sub!(CLIENT_REQUIRE, '//\0')
    dev_server_asset
  else
    File.read(file_path(logical_path))
  end
end

My solution was simply to remove the protocol and host/port from loaded asset path. If the protocol and host/port aren't included nothing will change, and if they are included they will be removed so that when they are added in the open they aren't duplicated. I hope you find this solution satisfactory.

I have submitted Pull Request #834

BookOfGreg · 2017-11-06T22:08:51Z

Fixed by #834 Thanks @joeyparis ! 💙

It solves encoding issue when polyfill source code is loaded before Encoding.default_external is set

randyv12 · 2017-11-30T17:25:53Z

Thank you guys!!

randyv12 changed the title ~~Question: local manifest.json for server-side rendering~~ Question: Webpacker integration -> using local manifest.json for server-side rendering Jun 9, 2017

gauravtiwari mentioned this issue Jun 15, 2017

Proposal to merge in changes from Webpacker Lite. rails/webpacker#464

Closed

BookOfGreg added the enhancement label Nov 3, 2017

BookOfGreg self-assigned this Nov 3, 2017

BookOfGreg added this to the 2.4.1 milestone Nov 3, 2017

This was referenced Nov 6, 2017

🐛 Fix bug that prevents loading assets with webpacker-dev-server joeyparis/react-rails#1

Merged

🐛 Fix bug that prevents loading assets with webpacker-dev-server #834

Merged

BookOfGreg closed this as completed Nov 6, 2017

BookOfGreg referenced this issue in ViliusLuneckas/react-rails Nov 28, 2017

Load polyfill source codes on demand

9fcad03

It solves encoding issue when polyfill source code is loaded before Encoding.default_external is set

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Question: Webpacker integration -> using local manifest.json for server-side rendering #739

Question: Webpacker integration -> using local manifest.json for server-side rendering #739

randyv12 commented Jun 9, 2017 •

edited

Loading

justin808 commented Jun 13, 2017

randyv12 commented Jun 14, 2017 •

edited

Loading

justin808 commented Jul 26, 2017

joeyparis commented Nov 3, 2017

BookOfGreg commented Nov 3, 2017 •

edited

Loading

joeyparis commented Nov 3, 2017 •

edited

Loading

BookOfGreg commented Nov 3, 2017

joeyparis commented Nov 3, 2017

BookOfGreg commented Nov 3, 2017

joeyparis commented Nov 3, 2017

joeyparis commented Nov 6, 2017

joeyparis commented Nov 6, 2017

BookOfGreg commented Nov 6, 2017

randyv12 commented Nov 30, 2017

Question: Webpacker integration -> using local manifest.json for server-side rendering #739

Question: Webpacker integration -> using local manifest.json for server-side rendering #739

Comments

randyv12 commented Jun 9, 2017 • edited Loading

justin808 commented Jun 13, 2017

randyv12 commented Jun 14, 2017 • edited Loading

justin808 commented Jul 26, 2017

joeyparis commented Nov 3, 2017

BookOfGreg commented Nov 3, 2017 • edited Loading

joeyparis commented Nov 3, 2017 • edited Loading

BookOfGreg commented Nov 3, 2017

joeyparis commented Nov 3, 2017

BookOfGreg commented Nov 3, 2017

joeyparis commented Nov 3, 2017

joeyparis commented Nov 6, 2017

joeyparis commented Nov 6, 2017

BookOfGreg commented Nov 6, 2017

randyv12 commented Nov 30, 2017

randyv12 commented Jun 9, 2017 •

edited

Loading

randyv12 commented Jun 14, 2017 •

edited

Loading

BookOfGreg commented Nov 3, 2017 •

edited

Loading

joeyparis commented Nov 3, 2017 •

edited

Loading