Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[performance] Increase performance of adding files to squashed layer #68

Merged
merged 1 commit into from
Mar 29, 2016
Merged

[performance] Increase performance of adding files to squashed layer #68

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions docker_scripts/image.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -509,6 +509,8 @@ def _squash_layers(self, layers_to_squash, layers_to_move):
with tarfile.open(self.squashed_tar, 'w', format=tarfile.PAX_FORMAT) as squashed_tar:
to_skip = []
missed_markers = {}
# List of filenames in the squashed archive
squashed_files = []

for layer_id in layers_to_squash:
layer_tar_file = os.path.join(
Expand All @@ -522,7 +524,6 @@ def _squash_layers(self, layers_to_squash, layers_to_move):
# We need the list of marker files upfront, so we can
# skip unnecessary files
markers = self._marker_files(layer_tar)
squashed_files = squashed_tar.getnames()

# Iterate over the marker files found for this particular
# layer and if in the squashed layers file corresponding
Expand All @@ -545,9 +546,6 @@ def _squash_layers(self, layers_to_squash, layers_to_move):
"Skipping '%s' file because it's on the list to skip files" % member.name)
continue

# List of filenames in the squashed archive
squashed_files = squashed_tar.getnames()

# Check if file is already added to the archive
if member.name in squashed_files:
# File already exist in the squashed archive, skip it because
Expand All @@ -566,6 +564,9 @@ def _squash_layers(self, layers_to_squash, layers_to_move):
squashed_tar.addfile(
member, layer_tar.extractfile(member))

# We added a file to the squashed tar, so let's note it
squashed_files.append(member.name)

self._add_markers(
missed_markers, squashed_tar, layers_to_move, self.old_image_dir)

Expand Down