Skip to content

Commit

Permalink
Merge pull request #29 from lordnull/mw-use-maps
Browse files Browse the repository at this point in the history
Use Maps as Json representation
  • Loading branch information
lordnull authored Oct 29, 2017
2 parents 2b2b417 + 559d4e4 commit 740d502
Show file tree
Hide file tree
Showing 14 changed files with 481 additions and 695 deletions.
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
language: erlang
otp_release:
- 18.0
- 17.0
- 20.0
- 19.0

script: make tests
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
PROJECT = rec2json
PROJECT_DESCRIPTION = Compile erlang record definitions into modules to convert them to/from json easily.
PROJECT_VERSION = 3.2.1
PROJECT_VERSION = 4.0.0

TEST_DEPS = proper jsx
dep_proper = git https://github.com/manopapad/proper master
Expand Down
98 changes: 44 additions & 54 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,22 @@

## Overview

Rec2json is a parse transform that takes a module which defines a record of
the same name and adds to_json, from_json, and introspection functions. The
to_json and from_json convert to and from the
[proposed erlang json standard](http://www.erlang.org/eeps/eep-0018.html)
Rec2json is a parse transform that takes a module which defines a record of
the same name and adds to_json, from_json, and introspection functions. The
to_json and from_json convert to and from the map based format used by jsx and
other json encoding and decoding libraries.

## Features

* Resulting modules can be used as parameterized modules or pure erlang.
* Uses a parse transform.
* Limited type checking on json -> record conversion.
* Limited type checking can include user defined types.
* Type checking on json -> record conversion using primitive built-in types.
* Type checking can be extended to include user defined types.
* Atom 'undefined' fields in records optionally skipped or set to null.
* Atom 'null' in json optionally converted to 'undefined'.
* Post processing options on record -> json convertion.
* Seed json -> record conversion with a record.
* Nested json -> record and record -> json conversions for other
records that have been compiled using rec2json.
* Nested json -> record and record -> json conversions for other records
that have been compiled using rec2json.
* Generated module has accessor functions for fields and field_names for
list of fields in the record.
* Above feature can be surpressed, and is careful by default.
Expand All @@ -46,13 +45,11 @@ module or in an include file.

The transformed modules depend on the rec2json application's modules.

The records are unchanged and can be used normally. A record can also be
used as a paramterized module, making it simple to use with the
[erlydtl Django templates](https://github.com/evanmiller/erlydtl) project.
The records are unchanged and can be used normally.

Options are passed to the rec2json parse transform through compile options.
The parse transform checks for the key 'rec2json' in the compile options.
The value is expected to be a proplist.
The value is expected to be a proplist.

Options can also be passed in on a per-module basis by adding one or more
`rec2json` module attributes. A `rec2json` module attribute can either be a
Expand All @@ -73,21 +70,21 @@ created. If set to false, they are not created nor exported.</td>
<tr>
<td>generate_setters</td> <td>true : boolean()</td> <td> If set to
true, functions for setting the fields of a record are created and
exported. These are of the form Field(NewVal, Record) or
Record:Field(NewVal). If set to false, they are not created nor exported.
exported. These are of the form Field(NewVal, Record). If set to false, they
are not created nor exported.
</td>
</tr>
<tr>
<td>careful</td> <td>true : boolean()</td> <td> If set to true,
rec2json's parse transform avoid altering or adding functions that are
rec2json's parse transform avoids altering or adding functions that are
already defined in the module. This means you can override the default
to_json/1 function to call to_json/2 with a specific set of options.</td>
</tr>
<tr>
<td>generate_type</td> <td>true : boolean()</td> <td> If set to
true, a type is generated for the record, and that type is exported. In
addition, a function is generated so that other rec2json records using the
exported type work as expected</td>
exported type work as expected.</td>
</tr>
<tr>
<td>type_name</td> <td> ?MODULE : atom()</td> <td> If generate_type is
Expand All @@ -112,9 +109,8 @@ Record = #person{ name = <<"John">>, age = 32, spouse = undefined }.
To convert a record to a json structure:

```erlang
Json = Record:to_json().
Json = person:to_json(Record).
[{name, <<"John">>}, {age, 32}] = Json.
#{name := <<"John">>, age := 32} = Json.
```

The to_json function can take a list of mutators. Mutators are applied in
Expand All @@ -123,36 +119,34 @@ the order listed except {null_is_undefined}. Supported mutators are:
* Turn undefined into null instead of skipping the field

```erlang
Record:to_json([{null_is_undefined}]).
person:to_json(Record, [{null_is_undefined}]).
```

* Add a property

```erlang
Record:to_json([{single, true}]).
person:to_json(Record, [{single, true}]).
person:to_json(Record, [#{single => true}).
person:to_json(Record, [#{single => true, employed => true}).
```

* Remove a property

```erlang
Record:to_json([age]).
person:to_json(Record, [age]).
```

* Modify based only on the json

```erlang
ModFunc = fun(Json) ->
case proplists:get_value(spouse, Json) of
undefined ->
[{single, true} | Json];
case maps:find(spouse, Json) of
error ->
Json#{single => true}
_ ->
[{single, false} | Json]
Json#{single => false}
end
end.
Record:to_json([ModFunc]).
person:to_json(Record, [ModFunc]).
```

Expand All @@ -162,12 +156,11 @@ person:to_json(Record, [ModFunc]).
ModFunc = fun(Json, Record) ->
case Record#person.spouse of
undefined ->
[{single, true} | Json];
Json#{single => true};
_ ->
[{single, false} | Json]
Json#{single => false}
end
end.
Record:to_json([ModFunc]).
person:to_json(Record, [ModFunc]).
```

Expand All @@ -176,52 +169,45 @@ person:to_json(Record, [ModFunc]).
Converting from a json structure to a record is just as simple:

```erlang
{ok, Record} = person:from_json([
{<<"name">>, <<"John">>},
{<<"age">>, 32},
{<<"spouse">>, null}
]).
{ok, Record} = person:from_json(#{
<<"name">> => <<"John">>,
<<"age">> => 32,
<<"spouse">> => null
}).
```

It may be desireable to change 'null' into 'undefined' in the record:
It may be desirable to change 'null' into 'undefined' in the record:

```erlang
{ok, Record} = person:from_json(Json, [null_is_undefined]).
```

It may be desireable to start with an existing record instead of creating
It may be desirable to start with an existing record instead of creating
a new one:

```erlang
{ok, Record2} = Record:from_json(Json).
{ok, Record2} = person:from_json(Json, Record).
{ok, Record2} = person:from_json(Record, Json).
{ok, Record2} = Record:from_json(Json, [null_is_undefined]).
{ok, Record2} = person:from_json(Record, Json, [null_is_undefined]).
```

If the json structure has a type that connot be reconciled with a type
If the json structure has a type that cannot be reconciled with a type
specified by the record definition, a list of fields with possible errors
is returned. The record will have the data that was in the json structure.
An untyped record field is the same as having the type 'any()'. There are
no warings about missing properties in the json, they simply retain the
no warnings about missing properties in the json, they simply retain the
default value of the record.

```erlang
{ok, Record, [age]} = person:from_json([{<<"age">>, <<"32">>}]).
{ok, Record, [age]} = person:from_json(#{<<"age">> => <<"32">>}).
```

### Including in a project

If all you are using is the parse_transform, simply add rec2json as a
required application.

To be able to create modules from records without the
parse transform, you will need to add the rec2json script to your path in some
manner. Add a call to the rec2json script during your build (in your Makefile
or rebar.config precompile hook).

## Type Checking and Converstion
## Type Checking and Conversion

Type conversion attempts to be as transparent and intuitive as possible.
There are some types that json does not represent directly, and some types
Expand All @@ -230,7 +216,7 @@ that have additional checking implemented.
Record fields that have atoms as types will have binary values in the json
checked. If the atom converted to the binary is equal to the json value, the
atom value is put into the record. When converting a record to json, atom
values will be converted to binaries to conform to the erlang spec.
values will be converted to binaries.

Lists have their types checked. If there is an invalid type, the invalid
type is placed in the list, but the warning message has the index of the
Expand All @@ -242,7 +228,7 @@ invalid type placed in the warning path list. For example:
}).

type_mismatch() ->
Json = [{ids, [<<"invalid">>, 3]}],
Json = #{ids => [<<"invalid">>, 3]},
{ok, Record, Warnings} = list_holder:from_json(Json),
#list_holder{ids = [<<"invalid">>, 3]} = Record,
[[ids, 1]] = Warnings.
Expand All @@ -262,7 +248,7 @@ list has the field name prepended to each. For example:
}).

type_mismatch() ->
Json = [{in_field, [{count, <<"0">>}]}],
Json = #{in_field => #{count => <<"0">>}},
{ok, Record, Warnings} = outer:from_json(Json),
#outer{in_field = #inner{ count = <<"0">> } } = Record,
[[in_filed, count]] = Warnings.
Expand Down Expand Up @@ -295,9 +281,9 @@ is to convert untrusted data, such as an http post request. Converting
untrusted data into atoms can exhaust the erlang vm's atom table, or worse
exhaust the machine's memory. Rec2json is, by default, safe.

It is still possilbe to apply a type to a record field so that json strings
It is still possible to apply a type to a record field so that json strings
will be converted to the equivalent atom using either a user defined type,
or the provied `r2j_type:unsafe_atom()` type.
or the provided `r2j_type:unsafe_atom()` type.

### User defined types

Expand All @@ -313,6 +299,10 @@ For example, given the module:
-module(type_example).
-compile([{parse_transform, rec2json}]).

% These two lines exist to satisfy dialyzer.
-type point() :: {number(), number()}.
-export_type([point/0]).

-record(type_example, {
some_field :: module:function(arg1, arg2),
xy = {0, 0} :: type_example:point()
Expand Down Expand Up @@ -342,4 +332,4 @@ error is thrown.

## Contributing

Fork and submit a pull request with relevent tests.
Fork and submit a pull request with relevant tests.
2 changes: 1 addition & 1 deletion ebin/rec2json.app
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{application, rec2json, [
{description, "Compile erlang record definitions into modules to convert them to/from json easily."},
{vsn, "3.2.1"},
{vsn, "4.0.0"},
{modules, ['r2j_compile','r2j_type','rec2json']},
{registered, []},
{applications, [kernel,stdlib]}
Expand Down
Loading

0 comments on commit 740d502

Please sign in to comment.