Added LayoutLMv3 #2178
Added LayoutLMv3 #2178
Conversation
|
@carrycooldude That you for the PR - the code structure does not match KerasHub style. |
| @@ -0,0 +1,152 @@ | |||
| """Tests for LayoutLMv3 backbone.""" | |||
There was a problem hiding this comment.
Remove these docstring at the start of the file.
|
Adding General code structuring comments.
Refer any existing model implementations here https://github.com/keras-team/keras-hub/tree/master/keras_hub/src/models The test cases also should follow the template we are following in the models. |
| @@ -0,0 +1 @@ | |||
|
|
|||
There was a problem hiding this comment.
Remove this directory and file
| @@ -0,0 +1,4 @@ | |||
| """LayoutLMv3 document classifier.""" | |||
There was a problem hiding this comment.
This file needs to be empty, all the import is handled in keras_hub/api directory and will be automatically generated whenever you run git commit -m "<message>"
Make sure you run pre-commit install for the first time.
| @@ -0,0 +1,15 @@ | |||
| from keras_hub.src.models.layoutlmv3.layoutlmv3_backbone import LayoutLMv3Backbone | |||
There was a problem hiding this comment.
This file is mainly to register presets, follow other models to understand the format we follow.
|
|
||
| def __init__( | ||
| self, | ||
| vocab_size: int = 30522, |
There was a problem hiding this comment.
Remove type annotation from everywhere, we don't follow type annotation in Keras Hub
| References: | ||
| - [LayoutLMv3 Paper](https://arxiv.org/abs/2204.08387) | ||
| - [LayoutLMv3 GitHub](https://github.com/microsoft/unilm/tree/master/layoutlmv3) | ||
| """ |
There was a problem hiding this comment.
This entire doctring needs to be inside the Backbone class
| """ | ||
|
|
||
| import os | ||
| from typing import Dict, List, Optional, Tuple, Union |
There was a problem hiding this comment.
Remove this once type annotation is removed
|
|
||
| from .layoutlmv3_tokenizer import LayoutLMv3Tokenizer | ||
| from .layoutlmv3_presets import backbone_presets | ||
| from .layoutlmv3_transformer import LayoutLMv3TransformerLayer |
There was a problem hiding this comment.
change from relative imports to absolute imports everywhere.
| maintaining spatial relationships in documents. | ||
|
|
||
| Args: | ||
| vocab_size: int, defaults to 30522. Size of the vocabulary. |
There was a problem hiding this comment.
Format for Args we follow is:
vocab_size: int. Size of the vocabulary. Defaults to 30522
This format should be followed for all and make sure it conveys the proper and complete required information.
| ``` | ||
| """ | ||
|
|
||
| presets = backbone_presets |
There was a problem hiding this comment.
No need of this here.
| self.use_rel_pos = use_rel_pos | ||
| self.rel_pos_bins = rel_pos_bins | ||
| self.max_rel_pos = max_rel_pos | ||
| self.spatial_embedding_dim = spatial_embedding_dim |
There was a problem hiding this comment.
This should come at last.
You can follow below order:
# === Layers ===
# === Functional Model ===
# === Config ===
Description
This PR fixes the LayoutLMv3 checkpoint conversion script to properly handle different spatial embedding dimensions between the base and large models. The base model uses 128 dimensions for all spatial embeddings, while the large model uses 171 dimensions for x/y coordinates and 170 dimensions for height/width.
Changes Made
Technical Details
The conversion script now:
Testing
Output Example