Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nhse d34 otp26 #1

Merged
merged 41 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
927cd19
Fix typo in atom
big-r81 Aug 11, 2022
a46b10a
Merge pull request #245 from big-r81/patch-1
etrepum Aug 11, 2022
38aadf3
Remove leading and trailing whitespaces
big-r81 Aug 11, 2022
16c43ef
Merge pull request #247 from big-r81/patch-2
etrepum Aug 14, 2022
48c0cdd
v3.1.0
etrepum Aug 14, 2022
6d22721
Merge pull request #248 from mochi/follow-up-247
etrepum Aug 14, 2022
bd0c848
fix release date
etrepum Aug 14, 2022
6d94ceb
Bug in mochiweb_request:read_chunk_length/2 #188
BradS2S Sep 21, 2022
1ccacbf
Merge pull request #249 from BradS2S/main
etrepum Sep 21, 2022
26f388b
Add OTP 25 to test matrix
etrepum Oct 7, 2022
a250223
Note latest compatibility in README
etrepum Oct 7, 2022
b1c5cca
Merge pull request #251 from mochi/otp-25
etrepum Oct 9, 2022
ef51f62
Update CHANGES.md
etrepum Oct 9, 2022
7ad3bd3
Add a test for the chunked encoding fix
etrepum Oct 12, 2022
7c4d311
Merge pull request #252 from mochi/test-chunked-encoding
etrepum Oct 12, 2022
7ed4ecf
Update README.md
vjc22 Feb 12, 2023
125d080
Update README.md
vjc22 Feb 12, 2023
eaba29b
Update README.md
vjc22 Feb 12, 2023
666ac57
Merge pull request #254 from vjc22/patch-1
etrepum Feb 13, 2023
5bd8411
Fix rebar edoc and test in CI
etrepum Apr 20, 2023
0733494
Merge pull request #257 from mochi/fix-edoc
etrepum Apr 20, 2023
8b2d844
Add mochiweb_request:is_closed/1 function
nickva Aug 31, 2023
af661f1
Merge pull request #258 from nickva/add-request-is-closed-function
etrepum Aug 31, 2023
f455126
Update CHANGES and version
etrepum Aug 31, 2023
cab4474
Merge pull request #259 from mochi/add-request-is-closed-function
etrepum Aug 31, 2023
1aa9ba1
mochinum:digits/1: fix handling of -0.0 for OTP-26.1/27.0
mikpe Sep 22, 2023
97573d2
Merge pull request #260 from mikpe/fix-match-on-float-zero
nickva Sep 22, 2023
897f22f
Update CHANGES for v3.2.1
etrepum Sep 22, 2023
fa3bf82
Update Erlang CI images
nickva Sep 22, 2023
293b95d
Update certificates
etrepum Sep 22, 2023
4e5372e
Merge pull request #261 from nickva/update-ci-images
etrepum Sep 22, 2023
a816606
use single quotes around 'maybe' atom for compatibility with OTP 27
michallepicki Mar 21, 2024
66a5ef8
Update CHANGES and ci matrix
etrepum Mar 21, 2024
4b03d12
Update mochiweb_util tests to avoid OTP 27 warnings
etrepum Mar 21, 2024
be68a43
Fix mochijson2 test that was map-order dependent
etrepum Mar 21, 2024
bc6ac7f
checkout v4
etrepum Mar 21, 2024
004080b
checkout@v3 for otp 18 & 19
etrepum Mar 21, 2024
611254e
Merge pull request #262 from michallepicki/otp-27
etrepum Mar 21, 2024
607c654
Update erlang.yml
martinsumner May 2, 2024
bf02c7d
Merge remote-tracking branch 'mochi/main' into nhse-d34-otp26
martinsumner May 2, 2024
278ef89
Test with OTP container
martinsumner May 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ insert_final_newline = true
[*.{erl,src,hrl}]
indent_style = space
indent_size = 4
tab_width = 8
28 changes: 16 additions & 12 deletions .github/workflows/erlang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,35 @@ name: Erlang CI

on:
push:
branches: [ develop ]
branches:
- nhse-develop
- nhse-develop-3.2
- nhse-develop-3.4
pull_request:
branches: [ develop ]
branches:
- nhse-develop
- nhse-develop-3.2
- nhse-develop-3.4


jobs:

build:

runs-on: ubuntu-latest
name: Test on ${{ matrix.os }} with OTP ${{ matrix.otp }}
runs-on: ${{ matrix.os }}

container:
image: erlang:${{matrix.otp}}

strategy:
fail-fast: false
matrix:
otp:
- "25.1"
- "24.3"
- "22.3"

container:
image: erlang:${{ matrix.otp }}
otp: [24, 26]
os: [ubuntu-latest]

steps:
- uses: lukka/get-cmake@latest
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Compile
run: ./rebar3 compile
- name: Run xref and dialyzer
Expand Down
36 changes: 36 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,39 @@
Version 3.2.2 released 2024-03-21

* Use single quotes around 'maybe' atom for compatibility with OTP 27
https://github.com/mochi/mochiweb/pull/262
* Update Erlang CI images
https://github.com/mochi/mochiweb/pull/261

Version 3.2.1 released 2023-09-22

* mochinum:digits/1: fix handling of -0.0 for OTP-26.1/27.0
https://github.com/mochi/mochiweb/pull/260

Version 3.2.0 released 2023-08-31

* Add new mochiweb_request:is_closed/1 function
https://github.com/mochi/mochiweb/pull/258

Version 3.1.2 released 2023-04-20

* Fix rebar edoc settings
https://github.com/mochi/mochiweb/pull/257

Version 3.1.1 released 2022-10-11

* OTP 25 added to test matrix
https://github.com/mochi/mochiweb/pull/251
* Fix for chunk length parsing edge case
https://github.com/mochi/mochiweb/pull/249

Version 3.1.0 released 2022-08-13

* Leading and trailing whitespace in header values are now trimmed
for better RFC 7230 compliance.
https://github.com/mochi/mochiweb/pull/247
https://github.com/mochi/mochiweb/pull/248

Version 3.0.0 released 2022-05-09

* rebar3 is now the preferred build tool (finally)
Expand Down
52 changes: 44 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,56 @@
# MochiWeb
MochiWeb is an Erlang library for building lightweight HTTP servers.

The latest version of MochiWeb is available at https://github.com/mochi/mochiweb
## Overview
MochiWeb provides a lightweight and fast solution for building HTTP servers in Erlang. The library provides features for building robust and scalable HTTP servers.

The mailing list for MochiWeb is at https://groups.google.com/group/mochiweb/
## Getting Started
Before you can use MochiWeb, you'll need to have [Erlang OTP](https://www.erlang.org/) installed. Once you have Erlang OTP installed, you can download the latest version of MochiWeb from the [GitHub repository](https://github.com/mochi/mochiweb).

Erlang OTP is required for setting up the MochiWeb environment and is available at https://www.erlang.org/
For a MochiWeb project, first obtain a copy of MochiWeb using Git by running the command.

To create a new mochiweb using project see the `example_project` in the `examples/` folder.

Information about Rebar (Erlang build tool) is available at https://github.com/erlang/rebar3
```erlang-repl
$ git clone git://github.com/mochi/mochiweb.git.
```
To create a project.

MochiWeb is currently tested with Erlang/OTP 18.3 through 24.0,
versions older than 3.0.0 may still be compatible back to R15B-03.
```erlang-repl
$ cd mochiweb
$ make app PROJECT=exampleName
```

# OTP 21.2, 21.2.1, 21.2.2 warning
You can now start the project with.

```erlang-repl
$ cd ../exampleName/
$ make
$ ./start-dev.sh
```

You can access the app by navigating to http://localhost:8080 in your browser.

For an example, view the `example_project` in the `examples/` folder.


## Benefits
* Lightweight: MochiWeb is designed to be lightweight and fast, making it ideal for use in resource-constrained environments.

* Easy to use: MochiWeb provides a simple and intuitive API that makes it easy to get started with building HTTP servers.

* Robust: MochiWeb provides a comprehensive set of features for building robust and scalable HTTP servers.

## Documentations and Resources
[Information about Rebar (Erlang build tool)](https://github.com/erlang/rebar3)

[Mailing list](https://groups.google.com/group/mochiweb/)

## OTP 21.2, 21.2.1, 21.2.2 warning

OTP 21.2 (up to and including 21.2.2) introduced an SSL regression that
makes these releases unsafe to use. See [ERL-830](https://bugs.erlang.org/browse/ERL-830).
This issue was resolved in OTP 21.2.3.


## Contributing
MochiWeb is an open-source project and welcomes contributions from the community.
5 changes: 5 additions & 0 deletions erlang_ls.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apps_dirs:
- "_build/default/lib/*"
include_dirs:
- "_build/default/lib/*/include"
- "include"
1 change: 1 addition & 0 deletions rebar.config
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
{test, [{deps, [meck]}]},
{gha, [{erl_opts, [{d, 'GITHUBEXCLUDE'}]}]}
]}.
{edoc_opts, [{preprocess, true}]}.
{dialyzer_opts, [{warnings, [no_return,
no_unused,
no_improper_lists,
Expand Down
5 changes: 3 additions & 2 deletions src/mochijson2.erl
Original file line number Diff line number Diff line change
Expand Up @@ -966,8 +966,9 @@ decode_map_test() ->
?assertEqual(M, decode(Json, [{format, map}])).

encode_map_test() ->
M = <<"{\"a\":1,\"b\":{\"c\":2}}">>,
?assertEqual(M, iolist_to_binary(encode(#{a => 1, b => #{ c => 2}}))).
?assertEqual(<<"{\"a\":1}">>, iolist_to_binary(encode(#{a => 1}))),
M = #{<<"a">> => 1, <<"b">> => #{<<"c">> => 2}},
?assertEqual(M, decode(iolist_to_binary(encode(#{a => 1, b => #{ c => 2}})), [{format, map}])).

encode_empty_map_test() ->
?assertEqual(<<"{}">>, encode(#{})).
Expand Down
4 changes: 3 additions & 1 deletion src/mochinum.erl
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
%% human-readable output, or compact ASCII serializations for floats.
digits(N) when is_integer(N) ->
integer_to_list(N);
digits(0.0) ->
digits(Float) when Float == 0.0 ->
"0.0";
digits(Float) ->
{Frac1, Exp1} = frexp_int(Float),
Expand Down Expand Up @@ -287,6 +287,8 @@ digits_test() ->
digits(0)),
?assertEqual("0.0",
digits(0.0)),
?assertEqual("0.0",
digits(-0.0)),
?assertEqual("1.0",
digits(1.0)),
?assertEqual("-1.0",
Expand Down
2 changes: 1 addition & 1 deletion src/mochiweb.app.src
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
%% This is generated from src/mochiweb.app.src
{application, mochiweb,
[{description, "MochiMedia Web Server"},
{vsn, "3.0.0"},
{vsn, "3.2.2"},
{modules, []},
{registered, []},
{env, []},
Expand Down
18 changes: 14 additions & 4 deletions src/mochiweb_headers.erl
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ lookup(K, T) ->
%% @doc Insert the pair into the headers if it does not already exist.
default(K, V, T) ->
K1 = normalize(K),
V1 = any_to_list(V),
V1 = trim_leading_and_trailing_ws(any_to_list(V)),
try gb_trees:insert(K1, {K, V1}, T)
catch
error:{key_exists, _} ->
Expand All @@ -185,15 +185,15 @@ default(K, V, T) ->
%% @doc Insert the pair into the headers, replacing any pre-existing key.
enter(K, V, T) ->
K1 = normalize(K),
V1 = any_to_list(V),
V1 = trim_leading_and_trailing_ws(any_to_list(V)),
gb_trees:enter(K1, {K, V1}, T).

%% @spec insert(key(), value(), headers()) -> headers()
%% @doc Insert the pair into the headers, merging with any pre-existing key.
%% A merge is done with Value = V0 ++ ", " ++ V1.
insert(K, V, T) ->
K1 = normalize(K),
V1 = any_to_list(V),
V1 = trim_leading_and_trailing_ws(any_to_list(V)),
try gb_trees:insert(K1, {K, V1}, T)
catch
error:{key_exists, _} ->
Expand All @@ -215,6 +215,9 @@ tokenize_header_value(undefined) ->
tokenize_header_value(V) ->
reversed_tokens(trim_and_reverse(V, false), [], []).

trim_leading_and_trailing_ws(S) ->

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since otp20 there is string:trim(S, both) that would help you a long way...
But possibly some other guarantees are given here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there may be a difference in the handling of "\r", which isn't referred to in this function. For now, I'm preferring to de-risk the change and not make any assumptions.

trim_and_reverse(trim_and_reverse(S, false), false).

trim_and_reverse([S | Rest], Reversed) when S=:=$ ; S=:=$\n; S=:=$\t ->
trim_and_reverse(Rest, Reversed);
trim_and_reverse(V, false) ->
Expand Down Expand Up @@ -242,7 +245,7 @@ reversed_tokens([C | Rest], Token, Acc) when C=:=$ ;C=:=$\n;C=:=$\t;C=:=$, ->
reversed_tokens([C | Rest], Token, Acc) ->
reversed_tokens(Rest, [C | Token], Acc);
reversed_tokens(_, _, _) ->
undefeined.
undefined.

extract_quoted_string([], _Acc) ->
undefined;
Expand Down Expand Up @@ -364,6 +367,13 @@ set_cookie_test() ->
to_list(H)),
ok.

whitespace_headers_test() ->
%% Check RFC 7230 whitespace compliance
H = ?MODULE:make([{"X-Auth-Roles", " test, test2,test3, test4, test5 , test6 "}]),
?assertEqual(
[{"X-Auth-Roles", "test, test2,test3, test4, test5 , test6"}],
to_list(H)).

headers_test() ->
H = ?MODULE:make([{hdr, foo}, {"Hdr", "bar"}, {'Hdr', 2}]),
[{hdr, "foo, bar, 2"}] = ?MODULE:to_list(H),
Expand Down
12 changes: 6 additions & 6 deletions src/mochiweb_multipart.erl
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ feed_mp(body,
C1 = Callback({body, Data}),
feed_mp(headers,
State#mp{callback = C1(body_end), buffer = Rest});
{maybe, Start} ->
{'maybe', Start} ->
<<Data:Start/binary, Rest/binary>> = Buffer,
feed_mp(body,
read_more(State#mp{callback = Callback({body, Data}),
Expand Down Expand Up @@ -328,15 +328,15 @@ find_boundary(Prefix, Data) ->
{end_boundary, Skip, size(Prefix) + 4};
_ when size(Data) < PrefixSkip + 4 ->
%% Underflow
{maybe, Skip};
{'maybe', Skip};
_ ->
%% False positive
not_found
end;
{partial, Skip, Length}
when Skip + Length =:= size(Data) ->
%% Underflow
{maybe, Skip};
{'maybe', Skip};
_ -> not_found
end.

Expand Down Expand Up @@ -695,8 +695,8 @@ find_boundary_test() ->
{end_boundary, 1, 9} = find_boundary(B,
<<"!\r\n--X--\r\nRest">>),
not_found = find_boundary(B, <<"--X\r\nRest">>),
{maybe, 0} = find_boundary(B, <<"\r\n--X\r">>),
{maybe, 1} = find_boundary(B, <<"!\r\n--X\r">>),
{'maybe', 0} = find_boundary(B, <<"\r\n--X\r">>),
{'maybe', 1} = find_boundary(B, <<"!\r\n--X\r">>),
P = <<"\r\n-----------------------------160374543510"
"82272548568224146">>,
B0 = <<55, 212, 131, 77, 206, 23, 216, 198, 35, 87, 252,
Expand All @@ -705,7 +705,7 @@ find_boundary_test() ->
45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45,
45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 49, 54, 48, 51,
55, 52, 53, 52, 51, 53, 49>>,
{maybe, 30} = find_boundary(P, B0),
{'maybe', 30} = find_boundary(P, B0),
not_found = find_boundary(B, <<"\r\n--XJOPKE">>),
ok.

Expand Down
23 changes: 13 additions & 10 deletions src/mochiweb_request.erl
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@
-export([accepted_content_types/2,
accepts_content_type/2]).

-export([is_closed/1]).

-define(SAVE_QS, mochiweb_request_qs).

-define(SAVE_PATH, mochiweb_request_path).
Expand Down Expand Up @@ -710,7 +712,7 @@ parse_post({?MODULE,

%% @spec stream_chunked_body(integer(), fun(), term(), request()) -> term()
%% @doc The function is called for each chunk.
%% Used internally by read_chunked_body.
%% Used internally by stream_body.
stream_chunked_body(MaxChunkSize, Fun, FunState,
{?MODULE,
[_Socket, _Opts, _Method, _RawPath, _Version,
Expand Down Expand Up @@ -769,7 +771,7 @@ read_chunk_length({?MODULE,
[{packet,
raw}])),
Splitter = fun (C) ->
C =/= $\r andalso C =/= $\n andalso C =/= $\n
C =/= $\r andalso C =/= $\n andalso C =/= $\s
end,
{Hex, _Rest} = lists:splitwith(Splitter,
binary_to_list(Header)),
Expand Down Expand Up @@ -1145,11 +1147,12 @@ accept_header({?MODULE,
Value -> Value
end.

%%
%% Tests
%%
-ifdef(TEST).

-include_lib("eunit/include/eunit.hrl").

-endif.
%% @spec is_closed(request()) -> true | false | undefined
%% @doc Check if a request connection is closing or already closed. This may be
%% useful when processing long running request callbacks, when the client
%% disconnects after a short timeout. This function works on Linux, NetBSD,
%% OpenBSD, FreeBSD and MacOS. On other operating systems, like Windows for
%% instance, it will return undefined.
is_closed({?MODULE,
[Socket, _Opts, _Method, _RawPath, _Version, _Headers]}) ->
mochiweb_socket:is_closed(Socket).
Loading