Skip to content

Commit 1a34c69

Browse files
v-robv-rob
committed
Add Lua binary network serialization functions
1 parent 518ecd7 commit 1a34c69

File tree

6 files changed

+548
-46
lines changed

6 files changed

+548
-46
lines changed

‎doc/lua_api.md

+49-2
Original file line numberDiff line numberDiff line change
@@ -5411,6 +5411,9 @@ Utilities
54115411
* `minetest.colorspec_to_colorstring(colorspec)`: Converts a ColorSpec to a
54125412
ColorString. If the ColorSpec is invalid, returns `nil`.
54135413
* `colorspec`: The ColorSpec to convert
5414+
* `minetest.colorspec_to_colorint(colorspec)`: Converts a ColorSpec to integer
5415+
form. If the ColorSpec is invalid, returns `nil`.
5416+
* `colorspec`: The ColorSpec to convert
54145417
* `minetest.colorspec_to_bytes(colorspec)`: Converts a ColorSpec to a raw
54155418
string of four bytes in an RGBA layout, returned as a string.
54165419
* `colorspec`: The ColorSpec to convert
@@ -5419,8 +5422,8 @@ Utilities
54195422
* `width`: Width of the image
54205423
* `height`: Height of the image
54215424
* `data`: Image data, one of:
5422-
* array table of ColorSpec, length must be width*height
5423-
* string with raw RGBA pixels, length must be width*height*4
5425+
* array table of ColorSpec, length must be `width * height`
5426+
* string with raw RGBA pixels, length must be `width * height * 4`
54245427
* `compression`: Optional zlib compression level, number in range 0 to 9.
54255428
The data is one-dimensional, starting in the upper left corner of the image
54265429
and laid out in scanlines going from left to right, then top to bottom.
@@ -6830,6 +6833,50 @@ Misc.
68306833
* Example: `deserialize('print("foo")')`, returns `nil`
68316834
(function call fails), returns
68326835
`error:[string "print("foo")"]:1: attempt to call global 'print' (a nil value)`
6836+
* `minetest.encode_network(format, ...)`: Encodes numbers and strings in binary
6837+
format suitable for network transfer according to a format string.
6838+
* Each character in the format string corresponds to an argument to the
6839+
function. Possible format characters:
6840+
* `b`: Signed 8-bit integer
6841+
* `h`: Signed 16-bit integer
6842+
* `i`: Signed 32-bit integer
6843+
* `l`: Signed 64-bit integer
6844+
* `B`: Unsigned 8-bit integer
6845+
* `H`: Unsigned 16-bit integer
6846+
* `I`: Unsigned 32-bit integer
6847+
* `L`: Unsigned 64-bit integer
6848+
* `f`: Single-precision floating point number
6849+
* `s`: 16-bit size-prefixed string. Max 64 KB in size
6850+
* `S`: 32-bit size-prefixed string. Max 64 MB in size
6851+
* `z`: Null-terminated string. Cannot have embedded null characters
6852+
* `Z`: Verbatim string with no size or terminator
6853+
* ` `: Spaces are ignored
6854+
* Integers are encoded in big-endian format, and floating point numbers are
6855+
encoded in IEEE-754 format. Note that the full range of 64-bit integers
6856+
cannot be represented in Lua's doubles.
6857+
* If integers outside of the range of the corresponding type are encoded,
6858+
integer wraparound will occur.
6859+
* If a string that is too long for a size-prefixed string is encoded, it
6860+
will be truncated.
6861+
* If a string with an embedded null character is encoded as a null
6862+
terminated string, it is truncated to the first null character.
6863+
* Verbatim strings are added directly to the output as-is and can therefore
6864+
have any size or contents, but the code on the decoding end cannot
6865+
automatically detect its length.
6866+
* `minetest.decode_network(format, data, ...)`: Decodes numbers and strings
6867+
from a binary format created by `minetest.encode_network()` according to a
6868+
format string.
6869+
* The format string follows the same rules as `minetest.encode_network()`.
6870+
The decoded values are returned as individual values from the function.
6871+
* `Z` has special behavior; an extra argument has to be passed to the
6872+
function for every `Z` specifier denoting how many characters to read.
6873+
To read all remaining characters, use a size of `-1`.
6874+
* If the end of the data is encountered while still reading values from the
6875+
string, values of the correct type will still be returned, but strings of
6876+
variable length will be truncated, and numbers and verbatim strings will
6877+
use zeros for the missing bytes.
6878+
* If a size-prefixed string has a size that is greater than the maximum, it
6879+
will be truncated and the rest of the characters skipped.
68336880
* `minetest.compress(data, method, ...)`: returns `compressed_data`
68346881
* Compress a string of data.
68356882
* `method` is a string identifying the compression method to be used.

‎games/devtest/mods/unittests/misc.lua

+199
Original file line numberDiff line numberDiff line change
@@ -177,3 +177,202 @@ local function test_on_mapblocks_changed(cb, player, pos)
177177
end
178178
end
179179
unittests.register("test_on_mapblocks_changed", test_on_mapblocks_changed, {map=true, async=true})
180+
181+
unittests.register("test_encode_network", function()
182+
-- 8-bit integers
183+
assert(minetest.encode_network("bbbbbbb", 0, 1, -1, -128, 127, 255, 256) ==
184+
"\x00\x01\xFF\x80\x7F\xFF\x00")
185+
assert(minetest.encode_network("BBBBBBB", 0, 1, -1, -128, 127, 255, 256) ==
186+
"\x00\x01\xFF\x80\x7F\xFF\x00")
187+
188+
-- 16-bit integers
189+
assert(minetest.encode_network("hhhhhhhh",
190+
0, 1, 257, -1,
191+
-32768, 32767, 65535, 65536) ==
192+
"\x00\x00".."\x00\x01".."\x01\x01".."\xFF\xFF"..
193+
"\x80\x00".."\x7F\xFF".."\xFF\xFF".."\x00\x00")
194+
assert(minetest.encode_network("HHHHHHHH",
195+
0, 1, 257, -1,
196+
-32768, 32767, 65535, 65536) ==
197+
"\x00\x00".."\x00\x01".."\x01\x01".."\xFF\xFF"..
198+
"\x80\x00".."\x7F\xFF".."\xFF\xFF".."\x00\x00")
199+
200+
-- 32-bit integers
201+
assert(minetest.encode_network("iiiiiiii",
202+
0, 257, 2^24-1, -1,
203+
-2^31, 2^31-1, 2^32-1, 2^32) ==
204+
"\x00\x00\x00\x00".."\x00\x00\x01\x01".."\x00\xFF\xFF\xFF".."\xFF\xFF\xFF\xFF"..
205+
"\x80\x00\x00\x00".."\x7F\xFF\xFF\xFF".."\xFF\xFF\xFF\xFF".."\x00\x00\x00\x00")
206+
assert(minetest.encode_network("IIIIIIII",
207+
0, 257, 2^24-1, -1,
208+
-2^31, 2^31-1, 2^32-1, 2^32) ==
209+
"\x00\x00\x00\x00".."\x00\x00\x01\x01".."\x00\xFF\xFF\xFF".."\xFF\xFF\xFF\xFF"..
210+
"\x80\x00\x00\x00".."\x7F\xFF\xFF\xFF".."\xFF\xFF\xFF\xFF".."\x00\x00\x00\x00")
211+
212+
-- 64-bit integers
213+
assert(minetest.encode_network("llllll",
214+
0, 1,
215+
511, -1,
216+
2^53-1, -2^53) ==
217+
"\x00\x00\x00\x00\x00\x00\x00\x00".."\x00\x00\x00\x00\x00\x00\x00\x01"..
218+
"\x00\x00\x00\x00\x00\x00\x01\xFF".."\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"..
219+
"\x00\x1F\xFF\xFF\xFF\xFF\xFF\xFF".."\xFF\xE0\x00\x00\x00\x00\x00\x00")
220+
assert(minetest.encode_network("LLLLLL",
221+
0, 1,
222+
511, -1,
223+
2^53-1, -2^53) ==
224+
"\x00\x00\x00\x00\x00\x00\x00\x00".."\x00\x00\x00\x00\x00\x00\x00\x01"..
225+
"\x00\x00\x00\x00\x00\x00\x01\xFF".."\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"..
226+
"\x00\x1F\xFF\xFF\xFF\xFF\xFF\xFF".."\xFF\xE0\x00\x00\x00\x00\x00\x00")
227+
228+
-- Strings
229+
local max_16 = string.rep("*", 2^16 - 1)
230+
local max_32 = string.rep("*", 2^26)
231+
232+
assert(minetest.encode_network("ssss",
233+
"", "hello",
234+
max_16, max_16.."too long") ==
235+
"\x00\x00".. "\x00\x05hello"..
236+
"\xFF\xFF"..max_16.."\xFF\xFF"..max_16)
237+
assert(minetest.encode_network("SSSS",
238+
"", "hello",
239+
max_32, max_32.."too long") ==
240+
"\x00\x00\x00\x00".. "\x00\x00\x00\x05hello"..
241+
"\x04\x00\x00\x00"..max_32.."\x04\x00\x00\x00"..max_32)
242+
assert(minetest.encode_network("zzzz",
243+
"", "hello", "hello\0embedded", max_16.."longer") ==
244+
"\0".."hello\0".."hello\0".. max_16.."longer\0")
245+
assert(minetest.encode_network("ZZZZ",
246+
"", "hello", "hello\0embedded", max_16.."longer") ==
247+
"".."hello".."hello\0embedded"..max_16.."longer")
248+
249+
-- Spaces
250+
assert(minetest.encode_network("B I", 255, 2^31) == "\xFF\x80\x00\x00\x00")
251+
assert(minetest.encode_network(" B Zz ", 15, "abc", "xyz") == "\x0Fabcxyz\0")
252+
253+
-- Empty format strings
254+
assert(minetest.encode_network("") == "")
255+
assert(minetest.encode_network(" ", 5, "extra args") == "")
256+
end)
257+
258+
unittests.register("test_decode_network", function()
259+
local d
260+
261+
-- 8-bit integers
262+
d = {minetest.decode_network("bbbbb", "\x00\x01\x7F\x80\xFF")}
263+
assert(#d == 5)
264+
assert(d[1] == 0 and d[2] == 1 and d[3] == 127 and d[4] == -128 and d[5] == -1)
265+
266+
d = {minetest.decode_network("BBBBB", "\x00\x01\x7F\x80\xFF")}
267+
assert(#d == 5)
268+
assert(d[1] == 0 and d[2] == 1 and d[3] == 127 and d[4] == 128 and d[5] == 255)
269+
270+
-- 16-bit integers
271+
d = {minetest.decode_network("hhhhhh",
272+
"\x00\x00".."\x00\x01".."\x01\x01"..
273+
"\x7F\xFF".."\x80\x00".."\xFF\xFF")}
274+
assert(#d == 6)
275+
assert(d[1] == 0 and d[2] == 1 and d[3] == 257 and
276+
d[4] == 32767 and d[5] == -32768 and d[6] == -1)
277+
278+
d = {minetest.decode_network("HHHHHH",
279+
"\x00\x00".."\x00\x01".."\x01\x01"..
280+
"\x7F\xFF".."\x80\x00".."\xFF\xFF")}
281+
assert(#d == 6)
282+
assert(d[1] == 0 and d[2] == 1 and d[3] == 257 and
283+
d[4] == 32767 and d[5] == 32768 and d[6] == 65535)
284+
285+
-- 32-bit integers
286+
d = {minetest.decode_network("iiiiii",
287+
"\x00\x00\x00\x00".."\x00\x00\x00\x01".."\x00\xFF\xFF\xFF"..
288+
"\x7F\xFF\xFF\xFF".."\x80\x00\x00\x00".."\xFF\xFF\xFF\xFF")}
289+
assert(#d == 6)
290+
assert(d[1] == 0 and d[2] == 1 and d[3] == 2^24-1 and
291+
d[4] == 2^31-1 and d[5] == -2^31 and d[6] == -1)
292+
293+
d = {minetest.decode_network("IIIIII",
294+
"\x00\x00\x00\x00".."\x00\x00\x00\x01".."\x00\xFF\xFF\xFF"..
295+
"\x7F\xFF\xFF\xFF".."\x80\x00\x00\x00".."\xFF\xFF\xFF\xFF")}
296+
assert(#d == 6)
297+
assert(d[1] == 0 and d[2] == 1 and d[3] == 2^24-1 and
298+
d[4] == 2^31-1 and d[5] == 2^31 and d[6] == 2^32-1)
299+
300+
-- 64-bit integers
301+
d = {minetest.decode_network("llllll",
302+
"\x00\x00\x00\x00\x00\x00\x00\x00".."\x00\x00\x00\x00\x00\x00\x00\x01"..
303+
"\x00\x00\x00\x00\x00\x00\x01\xFF".."\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"..
304+
"\x00\x1F\xFF\xFF\xFF\xFF\xFF\xFF".."\xFF\xE0\x00\x00\x00\x00\x00\x00")}
305+
assert(#d == 6)
306+
assert(d[1] == 0 and d[2] == 1 and d[3] == 511 and
307+
d[4] == -1 and d[5] == 2^53-1 and d[6] == -2^53)
308+
309+
d = {minetest.decode_network("LLLLLL",
310+
"\x00\x00\x00\x00\x00\x00\x00\x00".."\x00\x00\x00\x00\x00\x00\x00\x01"..
311+
"\x00\x00\x00\x00\x00\x00\x01\xFF".."\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF"..
312+
"\x00\x1F\xFF\xFF\xFF\xFF\xFF\xFF".."\xFF\xE0\x00\x00\x00\x00\x00\x00")}
313+
assert(#d == 6)
314+
assert(d[1] == 0 and d[2] == 1 and d[3] == 511 and
315+
d[4] == 2^64-1 and d[5] == 2^53-1 and d[6] == 2^64 - 2^53)
316+
317+
-- Floating point numbers
318+
local enc = minetest.encode_network("fff",
319+
0.0, 123.456, -987.654)
320+
assert(#enc == 3 * 4)
321+
322+
d = {minetest.decode_network("fff", enc)}
323+
assert(#d == 3)
324+
assert(d[1] == 0.0 and d[2] > 123.45 and d[2] < 123.46 and
325+
d[3] > -987.66 and d[3] < -987.65)
326+
327+
-- Strings
328+
local max_16 = string.rep("*", 2^16 - 1)
329+
local max_32 = string.rep("*", 2^26)
330+
331+
d = {minetest.decode_network("ssss",
332+
"\x00\x00".."\x00\x05hello".."\xFF\xFF"..max_16.."\x00\xFFtoo short")}
333+
assert(#d == 4)
334+
assert(d[1] == "" and d[2] == "hello" and d[3] == max_16 and d[4] == "too short")
335+
336+
d = {minetest.decode_network("SSSSS",
337+
"\x00\x00\x00\x00".."\x00\x00\x00\x05hello"..
338+
"\x04\x00\x00\x00"..max_32.."\x04\x00\x00\x08"..max_32.."too long"..
339+
"\x00\x00\x00\xFFtoo short")}
340+
assert(#d == 5)
341+
assert(d[1] == "" and d[2] == "hello" and
342+
d[3] == max_32 and d[4] == max_32 and d[5] == "too short")
343+
344+
d = {minetest.decode_network("zzzz", "\0".."hello\0".."missing end")}
345+
assert(#d == 4)
346+
assert(d[1] == "" and d[2] == "hello" and d[3] == "missing end" and d[4] == "")
347+
348+
-- Verbatim strings
349+
d = {minetest.decode_network("ZZZZ", "xxxyyyyyzzz", 3, 0, 5, -1)}
350+
assert(#d == 4)
351+
assert(d[1] == "xxx" and d[2] == "" and d[3] == "yyyyy" and d[4] == "zzz")
352+
353+
-- Read past end
354+
d = {minetest.decode_network("bhilBHILf", "")}
355+
assert(#d == 9)
356+
assert(d[1] == 0 and d[2] == 0 and d[3] == 0 and d[4] == 0 and
357+
d[5] == 0 and d[6] == 0 and d[7] == 0 and d[8] == 0 and d[9] == 0.0)
358+
359+
d = {minetest.decode_network("ZsSzZ", "xx", 4, 4)}
360+
assert(#d == 5)
361+
assert(d[1] == "xx\0\0" and d[2] == "" and d[3] == "" and
362+
d[4] == "" and d[5] == "\0\0\0\0")
363+
364+
-- Spaces
365+
d = {minetest.decode_network("B I", "\xFF\x80\x00\x00\x00")}
366+
assert(#d == 2)
367+
assert(d[1] == 255 and d[2] == 2^31)
368+
369+
d = {minetest.decode_network(" B Zz ", "\x0Fabcxyz\0", 3)}
370+
assert(#d == 3)
371+
assert(d[1] == 15 and d[2] == "abc" and d[3] == "xyz")
372+
373+
-- Empty format strings
374+
d = {minetest.decode_network("", "some random data")}
375+
assert(#d == 0)
376+
d = {minetest.decode_network(" ", "some random data", 3, 5)}
377+
assert(#d == 0)
378+
end)

0 commit comments

Comments
 (0)