Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement buffer_size overrides #399

Merged
merged 6 commits into from
Feb 3, 2022
Merged

Conversation

nyurik
Copy link
Member

@nyurik nyurik commented Jan 27, 2022

Layer file changes

  • Added support for an optional min_buffer_size value, which is only used in case of a global override, but if set, it must be less than or equal to buffer_size.
  • Layer's buffer_size is now optional if min_buffer_size is set.
layer:
  id: housenumber
  # Only one of these is mandatory
  min_buffer_size: 1
  buffer_size: 8

Tileset file changes

  • Added support for an optional overrides section (similar to defaults).
    • Only supports optional overrides.buffer_size int value for now. If set, this value will override the buffer_size set in the layer. If layer has min_buffer_size, the largest of the two will be used.
  • Added support for per layer overrides as described in Add support for extendable layer schema #377. Only supports buffer_size and min_buffer_size
  • An environment variable TILE_BUFFER_SIZE can be used instead of overrides.buffer_size, and it takes precedence.

This example will load two layers. Mountain peak will be used as is, and house numbers's min buffer zoom is modified. The global override=0 will set mountain peaks buffer to 0 (because it has no min_buffer_size defined inside the layer file), but the house number buffer size will be 5.

tileset:
  layers:
    - mountain_peak/mountain_peak.yaml
    - file: housenumber/housenumber.yaml
      min_buffer_size: 5
  overrides:
    buffer_size: 0

Breaking

  • Remove deprecated tileset and layer properties (they have been deprecated for many years, and are not used)

See openmaptiles/openmaptiles#1345

Layer file changes:
* Added support for an optional `min_buffer_size` value, which is only used in case of a global override, but if set, it must be less than or equal to `buffer_size`.
* Layer's `buffer_size` is now optional if `min_buffer_size` is set.

Tileset file changes:
* Added support for an optional `overrides` section (similar to `defaults`).
* Currently only suppports optional `overrides.buffer_size` int value. If set, this value will override the `buffer_size` set in the layer.  If layer has `min_buffer_size`, the largest of the two will be used.
* An environment variable `TILE_BUFFER_SIZE` can be used instead of overrides.buffer_size, and takes precedent.
@nyurik nyurik requested a review from TomPohys January 28, 2022 18:54
@farfromrefug
Copy link

@nyurik this look pretty awesome!

openmaptiles/tileset.py Outdated Show resolved Hide resolved
return self.definition['layer']['buffer_size']
"""Each layer must have a default buffer size, with either `buffer_size` or `min_buffer_size` or both.
If both are set, buffer_size must be >= min_buffer_size.
min_buffer_size is only used when there is a global buffer size override,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that min_buffer_size is also used with per-layer override.

Suggested change
min_buffer_size is only used when there is a global buffer size override,
`min_buffer_size` is only used when there is a `buffer_size` override.

openmaptiles/tileset.py Outdated Show resolved Hide resolved
openmaptiles/tileset.py Outdated Show resolved Hide resolved
min_buffer_size is only used when there is a global buffer size override,
e.g. if global is set to 0, and layer's min_buffer_size is set to 4, the result is 4.
Per layer tileset overrides are allowed for both buffer_size and min_buffer_size.
Per layer overrides have higher priority than global overrides, but less than ENV var.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding of this sentence is that env. variable has higher priority than per-layer min_buffer_size override.

Suggested change
Per layer overrides have higher priority than global overrides, but less than ENV var.
Per layer `buffer_size` override has a higher priority than a global override, but less than ENV var.
Per layer `min_buffer_size` tileset override has a higher priority than the layer's `min_buffer_size`.
The resulting `min_buffer_size` is used if it is higher that the resulting `buffer_size`.

As far as I understand the code, the final buffer size of a layer is

max(
  first_of(
    TILE_BUFFER_SIZE environment variable,
    layer-specific override of the `buffer_size`,
    global override of the `buffer_size`,
    layer's `buffer_size`,
    layer's min_buffer_size,
    0),
  first_of(
    layer-specific override of min_buffer_size,
    global override of min_buffer_size,
    layer's min_buffer_size )
)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly, the overrides follow this logic (or at least it should):

        max(
          first_found_value(
            TILE_BUFFER_SIZE env variable,
            buffer_size set in the tileset yaml file layer's section (per layer override),
            buffer_size set in the tileset yaml file at the top level (global override),
            buffer_size set in the layer yaml file,
            0),
          first_found_value(
            min_buffer_size set in the tileset yaml file layer's section (per layer override),
            min_buffer_size set in the layer yaml file,
            0)
        )

Note that there is no global min_buffer_size override as that has very little value IMO. Also, while there is a fallback to 0 for both, the layer file requires either one to be present. Lastly, both vars are computed independently until the very end.

Copy link
Member Author

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried to cleanup buffer size logic/docs, thx for feedback!

min_buffer_size is only used when there is a global buffer size override,
e.g. if global is set to 0, and layer's min_buffer_size is set to 4, the result is 4.
Per layer tileset overrides are allowed for both buffer_size and min_buffer_size.
Per layer overrides have higher priority than global overrides, but less than ENV var.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly, the overrides follow this logic (or at least it should):

        max(
          first_found_value(
            TILE_BUFFER_SIZE env variable,
            buffer_size set in the tileset yaml file layer's section (per layer override),
            buffer_size set in the tileset yaml file at the top level (global override),
            buffer_size set in the layer yaml file,
            0),
          first_found_value(
            min_buffer_size set in the tileset yaml file layer's section (per layer override),
            min_buffer_size set in the layer yaml file,
            0)
        )

Note that there is no global min_buffer_size override as that has very little value IMO. Also, while there is a fallback to 0 for both, the layer file requires either one to be present. Lastly, both vars are computed independently until the very end.

@nyurik nyurik requested a review from eva-j January 31, 2022 19:12
@nyurik nyurik merged commit aee912d into openmaptiles:master Feb 3, 2022
@nyurik nyurik deleted the buf-size-ovd branch February 3, 2022 17:03
francois2metz pushed a commit to indoorequal/openmaptiles-tools that referenced this pull request Feb 7, 2022
* implements openmaptiles/openmaptiles#1345
* partially implements openmaptiles#377

### Layer file changes
* Added support for an optional `min_buffer_size` value, which is only used in case of a global override, but if set, it must be less than or equal to `buffer_size`.
* Layer's `buffer_size` is now optional if `min_buffer_size` is set.

```yaml
layer:
  id: housenumber
  # Only one of these is mandatory
  min_buffer_size: 1
  buffer_size: 8
```

### Tileset file changes
* Added support for an optional `overrides` section (similar to `defaults`).
  * Only supports optional `overrides.buffer_size` int value for now. If set, this value will override the `buffer_size` set in the layer.  If layer has `min_buffer_size`, the largest of the two will be used.
* Added support for per layer overrides as described in openmaptiles#377. Only supports `buffer_size` and `min_buffer_size`
* An environment variable `TILE_BUFFER_SIZE` can be used instead of `overrides.buffer_size`, and it takes precedence.

This example will load two layers. Mountain peak will be used as is, and house numbers's min buffer zoom is modified. The global override=0 will set mountain peaks buffer to 0 (because it has no min_buffer_size defined inside the layer file), but the house number buffer size will be 5.

```yaml
tileset:
  layers:
    - mountain_peak/mountain_peak.yaml
    - file: housenumber/housenumber.yaml
      min_buffer_size: 5
  overrides:
    buffer_size: 0
```

### Breaking
* Remove deprecated tileset and layer properties (they have been deprecated for many years, and are not used)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants