Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS_CRT_BUILD_FORCE_STATIC_LIBS #596

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ set environment variable `AWS_CRT_BUILD_USE_SYSTEM_LIBS=1` while building from s
AWS_CRT_BUILD_USE_SYSTEM_LIBS=1 python3 -m pip install .
```

If these dependencies are available as both static and shared libs, you can force the static ones to be used by setting: `AWS_CRT_BUILD_FORCE_STATIC_LIBS=1`

## Mac-Only TLS Behavior

Please note that on Mac, once a private key is used with a certificate, that certificate-key pair is imported into the Mac Keychain. All subsequent uses of that certificate will use the stored private key and ignore anything passed in programmatically. Beginning in v0.6.2, when a stored private key from the Keychain is used, the following will be logged at the "info" log level:
Expand Down
65 changes: 35 additions & 30 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,14 +140,20 @@ def get_cmake_path():

def using_system_libs():
"""If true, don't build any dependencies. Use the libs that are already on the system."""
return os.getenv('AWS_CRT_BUILD_USE_SYSTEM_LIBS') == '1'
return (os.getenv('AWS_CRT_BUILD_USE_SYSTEM_LIBS') == '1'
or not os.path.exists(os.path.join(PROJECT_DIR, 'crt', 'aws-c-common', 'CMakeLists.txt')))


def using_system_libcrypto():
"""If true, don't build AWS-LC. Use the libcrypto that's already on the system."""
return using_system_libs() or os.getenv('AWS_CRT_BUILD_USE_SYSTEM_LIBCRYPTO') == '1'


def forcing_static_libs():
"""If true, force libs to be linked statically."""
return os.getenv('AWS_CRT_BUILD_FORCE_STATIC_LIBS') == '1'


class AwsLib:
def __init__(self, name, extra_cmake_args=[], libname=None):
self.name = name
Expand All @@ -156,12 +162,11 @@ def __init__(self, name, extra_cmake_args=[], libname=None):


# The extension depends on these libs.
# They're built along with the extension.
# They're built along with the extension (unless using_system_libs() is True)
AWS_LIBS = []
if sys.platform != 'darwin' and sys.platform != 'win32':
if not using_system_libcrypto():
# aws-lc produces libcrypto.a
AWS_LIBS.append(AwsLib('aws-lc', libname='crypto'))
# aws-lc produces libcrypto.a
AWS_LIBS.append(AwsLib('aws-lc', libname='crypto'))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this so that aws-lc is in AWS_LIBS even if we're not building it, in the same way that all the other non-OS libs are in AWS_LIBS even if we're not building them. This removes some weirdness later where we had to add crypto back into the list of libraries we need to link against

AWS_LIBS.append(AwsLib('s2n'))
AWS_LIBS.append(AwsLib('aws-c-common'))
AWS_LIBS.append(AwsLib('aws-c-sdkutils'))
Expand Down Expand Up @@ -296,9 +301,7 @@ def _build_dependencies(self):

def run(self):
if using_system_libs():
print("Skip building dependencies, using system libs.")
elif not os.path.exists(os.path.join(PROJECT_DIR, 'crt', 'aws-c-common', 'CMakeLists.txt')):
print("Skip building dependencies, source not found.")
print("Skip building dependencies")
else:
self._build_dependencies()

Expand Down Expand Up @@ -345,38 +348,40 @@ def awscrt_ext():
extra_link_args += ['-framework', 'Security']

else: # unix
# linker will prefer shared libraries over static if it can find both.
# force linker to choose static variant by using using
# "-l:libaws-c-common.a" syntax instead of just "-laws-c-common".
#
# This helps AWS developers creating Lambda applications from Brazil.
# In Brazil, both shared and static libs are available.
# But Lambda requires all shared libs to be explicitly packaged up.
# So it's simpler to link them in statically and have less runtime dependencies.
libraries = [':lib{}.a'.format(x) for x in libraries]
if forcing_static_libs():
# linker will prefer shared libraries over static if it can find both.
# force linker to choose static variant by using
# "-l:libaws-c-common.a" syntax instead of just "-laws-c-common".
#
# This helps AWS developers creating Lambda applications from Brazil.
# In Brazil, both shared and static libs are available.
# But Lambda requires all shared libs to be explicitly packaged up.
# So it's simpler to link them in statically and have less runtime dependencies.
#
# Don't apply this trick to dependencies that are always on the OS (e.g. librt)
libraries = [':lib{}.a'.format(x) for x in libraries]

# OpenBSD doesn't have librt; functions are found in libc instead.
if not sys.platform.startswith('openbsd'):
libraries += ['rt']

if using_system_libcrypto():
libraries += ['crypto']
else:
# hide the symbols from libcrypto.a
# this prevents weird crashes if an application also ends up using
# libcrypto.so from the system's OpenSSL installation.
extra_link_args += ['-Wl,--exclude-libs,libcrypto.a']

# OpenBSD 7.4+ defaults to linking with --execute-only, which is bad for AWS-LC.
# See: https://github.com/aws/aws-lc/blob/4b07805bddc55f68e5ce8c42f215da51c7a4e099/CMakeLists.txt#L44-L53
# (If AWS-LC's CMakeLists.txt removes these lines in the future, we can remove this hack here as well)
if sys.platform.startswith('openbsd'):
# hide the symbols from libcrypto.a
# this prevents weird crashes if an application also ends up using
# libcrypto.so from the system's OpenSSL installation.
# Do this even if using system libcrypto, since it could still be a static lib.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this out of the if-statement because:

Do this even if using system libcrypto, since it could still be a static lib.

extra_link_args += ['-Wl,--exclude-libs,libcrypto.a']

# OpenBSD 7.4+ defaults to linking with --execute-only, which is bad for AWS-LC.
# See: https://github.com/aws/aws-lc/blob/4b07805bddc55f68e5ce8c42f215da51c7a4e099/CMakeLists.txt#L44-L53
# (If AWS-LC's CMakeLists.txt removes these lines in the future, we can remove this hack here as well)
if sys.platform.startswith('openbsd'):
if not using_system_libcrypto():
extra_link_args += ['-Wl,--no-execute-only']

# FreeBSD doesn't have execinfo as a part of libc like other Unix variant.
# Passing linker flag to link execinfo properly
if sys.platform.startswith('freebsd'):
extra_link_args += ['-lexecinfo']
libraries += ['execinfo']
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this wasn't broken, it was just weird we were adding this library via extra_link_args instead of via libraries


# python usually adds -pthread automatically, but we've observed
# rare cases where that didn't happen, so let's be explicit.
Expand Down
Loading