Skip to content

Commit

Permalink
Custom LFS Sever (#970)
Browse files Browse the repository at this point in the history
We've created our own Git Large File Storage (LFS) server due to the
excessive cost of GitHub's bandwidth. Did you know all of their Ethernet
cables are actually gold coated diamonds?

This PR reverses the work done in #954 to compress large location JSON
files into a 7zip archive and handle the (de)compression of the archive.

Only JSON files within the `project/assets/database/` directory that are
larger than 5MB have been included in LFS. This translates to all of the
`looseLoot.json` files. The rest are small enough to be included in the
base repo.

A `.lfsconfig` file has been added to the root of the project to alert
git to the presence of the custom LFS server. This public server is
read-only. Write access is only available to developers within the
Single Player Tarkov GitHub organization.

<img
src="https://github.com/user-attachments/assets/7ddfec9b-5a9a-42e6-806d-fd419e4eaa4f"
width="250">
  • Loading branch information
refringe authored Dec 6, 2024
1 parent 2527823 commit 6c9e4fd
Show file tree
Hide file tree
Showing 107 changed files with 123,471 additions and 524,905 deletions.
22 changes: 20 additions & 2 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,4 +1,22 @@
# Normalize Line Endings
# Normalize Line Endings.
* text=auto eol=lf

project/assets/compressed/database/locations.7z filter=lfs diff=lfs merge=lfs -text
# LFS Tracking for large (5MB+) JSON files.
#
# Can be found with the following command:
# find project/assets/database -type f -name "*.json" -size +5M | while read file; do
# git lfs track "$file"
# done
project/assets/database/locations/bigmap/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/laboratory/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/interchange/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/factory4_night/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/shoreline/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/tarkovstreets/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/sandbox/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/sandbox_high/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/lighthouse/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/rezervbase/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/woods/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/locations/factory4_day/looseLoot.json filter=lfs diff=lfs merge=lfs -text
project/assets/database/templates/items.json filter=lfs diff=lfs merge=lfs -text
5 changes: 4 additions & 1 deletion .github/workflows/run-lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
lfs: true
lfs: false

- name: Checkout LFS
run: git lfs pull

- uses: actions/setup-node@v4
with:
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/run-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
lfs: true
lfs: false

- name: Checkout LFS
run: git lfs pull

- uses: actions/setup-node@v4
with:
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
*.exe
*.zip
*.d.ts
project/assets/database/locations/
project/build/
project/obj/
project/dist/
Expand Down
3 changes: 3 additions & 0 deletions .lfsconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[lfs]
url = https://lfs.sp-tarkov.com/sp-tarkov/server
locksverify = false
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,6 @@ The following commands are available after the initial setup. Run them with `npm
| `run:profiler` | Run the project in profiler mode. |
| `gen:types` | Generate types for the project. |
| `gen:docs` | Generate documentation for the project. |
| `database:compress` | Compress database files from working directory into archive. |
| `database:decompress` | Decompress database archives into working directory. |

### Debugging

Expand Down
3 changes: 0 additions & 3 deletions project/assets/compressed/database/locations.7z

This file was deleted.

32 changes: 32 additions & 0 deletions project/assets/database/locations/base.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"locations": {},
"paths": [{
"Source": "653e6760052c01c1c805532f",
"Destination": "5714dc692459777137212e12"
}, {
"Source": "5b0fc42d86f7744a585f9105",
"Destination": "5714dc692459777137212e12"
}, {
"Source": "5714dc692459777137212e12",
"Destination": "5714dbc024597771384a510d"
}, {
"Source": "5714dbc024597771384a510d",
"Destination": "56f40101d2720b2a4d8b45d6"
}, {
"Source": "56f40101d2720b2a4d8b45d6",
"Destination": "55f2d3fd4bdc2d5f408b4567"
}, {
"Source": "55f2d3fd4bdc2d5f408b4567",
"Destination": "5704e3c2d2720bac5b8b4567"
}, {
"Source": "5704e3c2d2720bac5b8b4567",
"Destination": "5704e5fad2720bc05b8b4567"
}, {
"Source": "5704e5fad2720bc05b8b4567",
"Destination": "5704e4dad2720bb55b8b4567"
}, {
"Source": "5704e4dad2720bb55b8b4567",
"Destination": "5704e554d2720bac5b8b456e"
}
]
}
Loading

0 comments on commit 6c9e4fd

Please sign in to comment.