Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
puigru committed Aug 1, 2020
0 parents commit de1d57c
Show file tree
Hide file tree
Showing 73,327 changed files with 440,123 additions and 0 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
other.7z/* linguist-language=YAML
47 changes: 47 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: CI

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: '3.x'
- name: Prepare
id: prepare
run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
- name: Install dependencies
run: pip install pyyaml
- name: Build tree
run: python build.py other.7z build.json
- uses: actions/upload-artifact@v2
with:
name: build
path: build.json
- name: Create release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ steps.prepare.outputs.sha_short }}
release_name: ${{ steps.prepare.outputs.sha_short }}
draft: false
prerelease: false
- name: Upload release
id: upload_release
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./build.json
asset_name: build.json
asset_content_type: application/json
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
__pycache__/
*.json
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2020 Joel Puig Rubio

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Gigaleak documentation

This repository aims to document every file found in the leaks of July 24th and 25th, 2020, commonly known as the "gigaleak".

The file structure of this repository mirrors that of the leaked archives. However, files instead contain metadata in YAML format.

This can then be built into a JSON file listing all of the files and their description in a tree-like fashion which can be useful when learning about these files.

## Contributing

Going through all these files is a very time-consuming task, so any contributions are welcome.

### Windows

Windows places [many restrictions](https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#naming-conventions) on allowed filenames. These archives often do not abide by these restrictions, so this repository cannot be checked out as-is.

I strongly recommend using the [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10) to avoid running into issues.

If you must, you can clone the repository as such:

* `git clone -n https://github.com/XXLuigiMario/gigaleak.git`

Make sure you have Python installed and the `pyyaml` package:

* `pip install -U pyyaml`

Download the [latest release](https://github.com/XXLuigiMario/gigaleak/releases/latest) to the repository folder, then:

* `git config core.sparseCheckout true`
* `(echo /* & echo !other.7z) >.git\info\sparse-checkout`
* `git checkout`
* `python expand.py build.json .`
* `echo /* >.git\info\sparse-checkout`
* `git read-tree HEAD`

Or alternatively:

*Note: This will not check out the rest of the repository.*

* `git checkout HEAD expand.py`
* `python expand.py build.json .`
* `git reset`

Despite errors, this should leave the repository in a usable state. If you check `git status`, you will see that any files which did not follow the aforementioned naming convention are marked as deleted.

Be mindful of only including modified files when staging your changes!

## Building

Clone the repository and then run:

* `pip install pyyaml`
* `./build.py other.7z build.json`
37 changes: 37 additions & 0 deletions build.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/usr/bin/env python3
import argparse
import json
import os

import yaml

def build_tree(source):
tree = dict()
source = os.path.normpath(source)
source_parent = os.path.dirname(source)
for root, dirs, files in os.walk(source):
print('Building:', root)
path = os.path.dirname(os.path.relpath(root, source_parent))
parent = tree
if path:
for entry in path.split(os.path.sep):
parent = parent[entry]
contents = dict()
files.remove('current_dir')
with open(os.path.join(root, 'current_dir'), 'r', encoding='utf-8') as f:
contents['.'] = yaml.load(f, Loader=yaml.FullLoader)
for fname in sorted(files):
with open(os.path.join(root, fname), 'r', encoding='utf-8') as f:
contents[fname] = yaml.load(f, Loader=yaml.FullLoader)
parent[os.path.basename(root)] = contents
dirs.sort()
return tree

if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('source')
parser.add_argument('filename')
args = parser.parse_args()
tree = build_tree(args.source)
with open(args.filename, 'w', encoding='utf-8') as f:
json.dump(tree, f, indent=4, ensure_ascii=False)
42 changes: 42 additions & 0 deletions expand.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/usr/bin/env python3
import argparse
import json
import os

import yaml

def expand(node, dest, path=None, fail_ok=False):
if path:
current = os.path.join(dest, path)
if '.' not in node:
with open(os.path.join(dest, path), 'x', encoding='utf-8') as f:
yaml.dump(node, f, allow_unicode=True, sort_keys=False)
return
os.mkdir(current)
for name, node in node.items():
if name == '.':
name = 'current_dir'
child = os.path.join(path, name) if path else name
try:
expand(node, dest, child, fail_ok)
# on Windows, opening "CON" for exclusive creation raises ValueError
except (OSError, ValueError):
print('\033[31mUnable to create:', child, '\033[0m')
if fail_ok:
continue
raise

if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('filename')
parser.add_argument('destination')
args = parser.parse_args()
with open(args.filename, 'r', encoding='utf-8') as f:
root = json.load(f)
if os.name == 'nt':
os.system('color')
print('\033[33mWindows places many restrictions on filenames.',\
'Please use WSL to avoid missing files.\033[0m')
expand(root, args.destination, fail_ok=True)
else:
expand(root, args.destination)
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: Disk1
Type: folder
Size: 0
Last-Modified: null
SHA-1: null
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: ISAS32.EXE
Type: file
Size: 193024
Last-Modified: '1998-10-15T03:00:00Z'
SHA-1: 70C7240B4A366CFE447D7E7BC8C792566503FC1D
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: ISLK32.EXE
Type: file
Size: 115712
Last-Modified: '1998-09-24T03:00:00Z'
SHA-1: 23ECA9EF8A6E33C2FB0F69C96C657AB519FC6A34
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: BACK
Type: folder
Size: 0
Last-Modified: null
SHA-1: null
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C.isx
Type: file
Size: 711830
Last-Modified: '1998-11-17T01:55:30Z'
SHA-1: 156B8968FCE0383A3F399C8C0EFDFB51BCDF2998
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: cgal.bat
Type: file
Size: 690
Last-Modified: '1998-11-10T02:37:30Z'
SHA-1: 0261F0E21D2B22456DB1E8825FECF1AB573600CB
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C1.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:39:24Z'
SHA-1: 0ACCEF92A38DA757FD6147A483320FD207A40D49
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C2.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:39:32Z'
SHA-1: FA26CFC26AEA0DFF9599792203A658CF5CE23D5A
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C3.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:39:38Z'
SHA-1: FA529F239A785F7BA6AF22C4AB2CA443A3792468
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C4.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:40:00Z'
SHA-1: 382A8D4CCAB0827BA013FB95B30429AE181E10F1
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C5.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:40:06Z'
SHA-1: 803DCEC4FF7CDBC718D5B69222287A8877048563
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C6.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:40:12Z'
SHA-1: 6C8A0E524CB16C43FD1F9A7F7C6AC32D8ABB72A0
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C7.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T02:51:44Z'
SHA-1: 28B3F488E3B89E308E08B9DBBC2B0C5C473CCCAC
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: C8.CHR
Type: file
Size: 16384
Last-Modified: '1993-06-24T01:40:30Z'
SHA-1: 938C2C9E3A70BCDE00C5F2964DC6CB9F9134688F
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: cgx
Type: folder
Size: 0
Last-Modified: null
SHA-1: null
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g1.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:57:22Z'
SHA-1: 0ACCEF92A38DA757FD6147A483320FD207A40D49
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g2.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:57:56Z'
SHA-1: FA26CFC26AEA0DFF9599792203A658CF5CE23D5A
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g3.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:58:14Z'
SHA-1: FA529F239A785F7BA6AF22C4AB2CA443A3792468
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g4.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:58:30Z'
SHA-1: 382A8D4CCAB0827BA013FB95B30429AE181E10F1
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g5.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:59:00Z'
SHA-1: 803DCEC4FF7CDBC718D5B69222287A8877048563
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g6.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:59:12Z'
SHA-1: 6C8A0E524CB16C43FD1F9A7F7C6AC32D8ABB72A0
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g7.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:59:26Z'
SHA-1: 28B3F488E3B89E308E08B9DBBC2B0C5C473CCCAC
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: g8.bin
Type: file
Size: 16384
Last-Modified: '1998-11-16T03:59:36Z'
SHA-1: 938C2C9E3A70BCDE00C5F2964DC6CB9F9134688F
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: isdwdcmd.dat
Type: file
Size: 51
Last-Modified: '1998-11-16T04:06:08Z'
SHA-1: 53E2BE7E58454443CA8DC9CF27DFD99691D60FC6
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: isdwdrng.dat
Type: file
Size: 29448
Last-Modified: '1998-11-16T04:06:10Z'
SHA-1: D9F5C6ADD7DF21375490B479ACFA1B2FE347DBFE
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: isdwdsym.dat
Type: file
Size: 271582
Last-Modified: '1998-11-16T04:06:10Z'
SHA-1: 6D149694D3EDA960231B944861A674B6011FC9DE
Description: null
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Name: clink.bat
Type: file
Size: 35
Last-Modified: '1998-11-10T02:41:50Z'
SHA-1: FAD91B1848962D9F5C84AAA6A73ADA800273A87B
Description: null
Loading

0 comments on commit de1d57c

Please sign in to comment.